Objectives

  1. End to end analysis using R
  2. Learn the caret package for ML
  3. Learn to present the case using R Notebooks

Read in the dataset

I stored the raw files on Github, so I used RCurl with Wehrley’s method that utilizes read.csv to the fullest. It’s one of the best ways I’ve found to read in data and also set data-types at the same time. He’s done a great job on that function. The dataset contains one ID variable, one response variable and ten predictor variables.

library(RCurl,quietly = T)
library(tidyverse,quietly = T)
library(ggplot2,quietly = T)
library(gridExtra,quietly = T)
library(Amelia,quietly = T)
library(beanplot,quietly = T)
library(caret,quietly = T)
library(stringr,quietly = T)
library(party, quietly = T)
# library(rattle, quietly = T)
readData <- function(path.name, file.name, column.types, missing.types) {
    gurl <- paste(path.name,file.name,sep="")
    download.file(gurl,file.name,method="curl",quiet = T)
    tbl_df(read.csv(file.name,colClasses=column.types,
             na.strings=missing.types))
}
Titanic.path <- "https://raw.githubusercontent.com/rsangole/Titanic/master/"
train.data.file <- "train.csv"
test.data.file <- "test.csv"
missing.types <- c("NA", "")
train.column.types <- c('integer',   # PassengerId
                        'factor',    # Survived
                        'factor',    # Pclass
                        'character', # Name
                        'factor',    # Sex
                        'numeric',   # Age
                        'integer',   # SibSp
                        'integer',   # Parch
                        'character', # Ticket
                        'numeric',   # Fare
                        'character', # Cabin
                        'factor'     # Embarked
)
test.column.types <- train.column.types[-2]     # # no Survived column in test.csv
train.raw <- readData(Titanic.path, train.data.file,train.column.types,missing.types)
test.raw <- readData(Titanic.path, test.data.file,test.column.types,missing.types)
prep_data <- function(D) {
    if (!is.null(D$Survived)) {
        D$Survived <- factor(D$Survived,
                             levels = c(1, 0),
                             labels = c('Survived', 'Dead'))
        }
    D$Pclass <- factor(D$Pclass,
                       levels = c(1, 2, 3),
                       labels = c('P1', 'P2', 'P3'))
    D$PassengerId <- NULL
    D
}
train.raw <- prep_data(train.raw)
test.raw <- prep_data(test.raw)
str(train.raw)
Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   891 obs. of  11 variables:
 $ Survived: Factor w/ 2 levels "Survived","Dead": 2 1 1 1 2 2 2 2 1 1 ...
 $ Pclass  : Factor w/ 3 levels "P1","P2","P3": 3 1 3 1 3 3 1 3 3 2 ...
 $ Name    : chr  "Braund, Mr. Owen Harris" "Cumings, Mrs. John Bradley (Florence Briggs Thayer)" "Heikkinen, Miss. Laina" "Futrelle, Mrs. Jacques Heath (Lily May Peel)" ...
 $ Sex     : Factor w/ 2 levels "female","male": 2 1 1 1 2 2 2 2 1 1 ...
 $ Age     : num  22 38 26 35 35 NA 54 2 27 14 ...
 $ SibSp   : int  1 1 0 1 0 0 0 3 0 1 ...
 $ Parch   : int  0 0 0 0 0 0 0 1 2 0 ...
 $ Ticket  : chr  "A/5 21171" "PC 17599" "STON/O2. 3101282" "113803" ...
 $ Fare    : num  7.25 71.28 7.92 53.1 8.05 ...
 $ Cabin   : chr  NA "C85" NA "C123" ...
 $ Embarked: Factor w/ 3 levels "C","Q","S": 3 1 3 3 3 2 3 3 3 1 ...

Missing values analysis

Quick investigation of missing values can be done using the complete.cases(), and more thorough graphical summary can be done using Amelia. Overall, 79% of the observations have some missing data.

#Complete cases (percentages)
round(prop.table(table(complete.cases(train.raw))),2)

FALSE  TRUE 
 0.79  0.21 

Amelia lets us graphically investigate which variables have missing data. purr::map_xxx() gives this same information numerically in a succint fashion.

missmap(train.raw, main='Missing Values Analysis using Amelia ordered by % missing', col=c('red', 'gray'),legend = F,rank.order = T)

#Missing cases (numbers):
map_int(train.raw,~sum(is.na(.x)))
Survived   Pclass     Name      Sex      Age    SibSp    Parch   Ticket     Fare    Cabin 
       0        0        0        0      177        0        0        0        0      687 
Embarked 
       2 
#Missing cases (percentages):
round(map_dbl(train.raw,~sum(is.na(.x))/length(.x)),2)
Survived   Pclass     Name      Sex      Age    SibSp    Parch   Ticket     Fare    Cabin 
    0.00     0.00     0.00     0.00     0.20     0.00     0.00     0.00     0.00     0.77 
Embarked 
    0.00 

Cabin has a large number of missing values (77% missing). Imputing this variable may prove challenging or even useless. Age (19.9% missing) and Embarked (0.2%) missing are much more managable.


EDA

The first step in the analysis is to explore the data numerically and graphically. I always split up my EDA investigation as follows:

  • Target Variable
  • Predictor Variables
    • Univariate
    • Bivariate
    • Multivariate

This gives me a structured approach towards larger datasets. My professor at Northwestern taught me to always complete a thorough intimate numeric & graphical EDA on the data, no matter how large the data 1. Anscombe (1973) clearly shows the importance of graphical analyses.

Target Variable

Survived is the response variable. As we can see, a large majority of the passengers did not survive the accident. The response variable is a False/True boolean variable. Thus, the analysis techniques used later will be those appropriate for classification problems.

round(prop.table(table(train.raw$Survived)),2)

Survived     Dead 
    0.38     0.62 

Predictor Variables

Univariate & Bivariate

The first step is to look at every variable available. I prefer using the ggplot2 framework for all the visuals.

Continuous Variables

  • Age seems to have a bimodal distribution - very young children, and then directly young adults to mid-age persons. The 2nd mode is right skewed with no obvious outliers.

  • Fare certainly shows many outliers beyond the ~$200 level. A majority of the fares are <$50, which makes sense since a majority of the travelers are bound to be in the 3rd passenger class.

p1 <- ggplot(data=train.raw,aes(x=Age))+geom_histogram(bins = 40)
p2 <- ggplot(data=train.raw,aes(x=Fare))+geom_histogram(bins = 40)
grid.arrange(p1,p2)

As we can see, the median fare is $14.5, the mean is $32, but the max is $512. We’ll investigate winzorising this variable in the latter part. Perhaps a transformation will also help?

summary(train.raw$Fare)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   0.00    7.91   14.45   32.20   31.00  512.33 

Categorical Variables

A ggplot command is iterated over for the categorical variables.2

Key takeways for the categorical variables:

  1. Pclass: If you were traveling 1st class, you have the highest chance of survival. Could be indicative of preferential treatment to those who paid more, a less politically correct class-stratified society, as well as the fact that the 1st class passengers had cabins at the very top of the ship.
  2. Pclass: Persons traveling 3rd class had the highest fatality rate. 3rd class passengers had cabins deep in the ship. With the reasons give in (1), this could have contributed to the low survival rate.
  3. Sex: Males have a very high fatality rate. Seems like the ‘women and children’ first policy was followed during evacuation.
  4. SibSp & Parch: What’s interesting here is, for both these variables, at level 0, the fatality rate is higher. At levels 1+, the chances of survival are much better. Again, this could point to the ‘women and children’ policy being followed. (Or perhaps there weren’t as many families with children on board!)
  5. Embarked: Southampton has a higher fatality rate than Cherbourg or Queenstown. A cross-tabulation between Embarked and Pclass shows that 72% of the 3rd class passengers and 89% of the 2nd class passengers boarded at Southampton. This jives with the observation that 2nd and 3rd class passengers have higher fatality rates.
get_legend<-function(myggplot){
  tmp <- ggplot_gtable(ggplot_build(myggplot))
  leg <- which(sapply(tmp$grobs, function(x) x$name) == "guide-box")
  legend <- tmp$grobs[[leg]]
  return(legend)
}
p <- lapply(X = c('Pclass','Sex','SibSp','Parch','Embarked'),
            FUN = function(x) ggplot(data = train.raw)+
                aes_string(x=x,fill='Survived')+
                geom_bar(position="dodge")+
                theme(legend.position="none"))
legend <- get_legend(ggplot(data = train.raw,aes(x=Pclass,fill=Survived))+geom_bar())
grid.arrange(p[[1]],p[[2]],p[[3]],p[[4]],p[[5]],legend,layout_matrix =
                 cbind(c(1,2,3),c(4,5,NA),c(6,6,6)),widths=c(3,3,1))

# round(prop.table(table(train.raw$Embarked,train.raw$Pclass),margin = 2),2)

Multivariate Analyses

Grouped boxplots are a common method of comparing distributions grouped by categorical variables. I find beanplots to be excellent complementary plots to boxplots (and in some cases, even better). They’re a bit tricky to read at first - since they are so underutilized - but just through one plot, a wealth of information can be extracted.3

Here is a comparison of the same information between a boxplot and a beanplot. What can we infer from the bean plot better?

  1. The beanplot allows us to visualize the density function of the parameter, in this case: Age. Furthermore, the length of each beanline is cumulative to the number of datapoints that exist. Rightaway, we can tell that Pclass=3 has the most data in the set, with sparser data at Pclass=1.
  2. The mean values for 1st class is higher than that for 2nd and 3rd class. The distributions of deceased and survived for 1st class are fairly similar.
  3. For 2nd and 3rd class, the survived data shows a bimodal distribution. Bumps at the 0-10 age show that children were evacuated first. This is also the reason the mean values for survived is lower.
  4. For 2nd and 3rd class, the deceased data shows a fairly normal distribution.
  5. The individual measurements (represented by black lines) represent each observation and help identify outliers much more easily than a boxplot does.
ggplot(train.raw,aes(y=Age,x=Pclass))+geom_boxplot(aes(fill=Survived))+theme_bw()

beanplot(Age~Survived*Pclass,side='b',train.raw,col=list('yellow','orange'),
         border = c('yellow2','darkorange'),ll = 0.05,boxwex = .5,
         main='Passenger survival by pclass and Age',xlab='Passenger Class',ylab='Age')
legend('topright', fill = c('yellow','orange'), legend = c("Dead", "Survived"),bty = 'n',cex = .8)

A look into the SibSp and Parch variables shows something interesting. There are three regions one can identify:

  • The probability of survival is minimal for number of parents/children aboard > 3.
  • The probability of survival is minimal for number of siblings/spouses aboard > 3.
  • For SibSp<=3 and Parch<=3, there are better chances for survival.

The grouping by Pclass reveals that all the large families were 3rd class travelers. Worse access to help… lowest chance for survival.

These could be simple rules either hard coded during model building: something along the lines of: IF (SibSp>3 OR Parch >3) THEN prediction = 0, or some derived variables can be created.

ggplot(train.raw,aes(y=SibSp,x=Parch))+
    geom_jitter(aes(color=Survived,shape=Pclass))+
    theme_bw()+
    scale_shape(solid=F)+
    geom_vline(xintercept = 3,color='darkblue',lty=3)+
    geom_hline(yintercept = 3,color='darkblue',lty=3)


Data Preparation

Missing Values Imputation

Starting with the easier one first:

Embarked: The largest portion of the passengers embared at Southhampton. I’m replacing the NAs with the same. First, I create a new imputed training dataset.

summary(train.raw$Embarked)
   C    Q    S NA's 
 168   77  644    2 
train.imp <- train.raw
train.imp$Embarked[is.na(train.imp$Embarked)]='S'

Names, Titles & Age:

The names have titles embedded in the strings. I can extract these using regex. Master, Miss, Mr and Mrs are the most popular - no surprise there, with lots of other titles. Here’s the distribution of the titles by age. These can be used to impute the missing age values.

train.raw$title <- str_extract(pattern = '[a-zA-Z]+(?=\\.)',string = train.raw$Name)
train.raw$title <- as.factor(train.raw$title)
train.raw %>%
    na.omit() %>%
    group_by(title) %>%
    dplyr::summarise(Count=n(), Median_Age=round(median(Age),0)) %>%
    arrange(-Median_Age)
ggplot(train.raw,aes(x=title,y=Age))+
    stat_summary(aes(y = Age,group=1), fun.y=median, colour="red", geom="point",group=1)+
    geom_jitter(shape=21,alpha=.6,col='blue')+
    theme_bw()+
    theme(axis.text.x = element_text(angle = 45, hjust = 1),legend.position="none")+
    labs(caption='Red points are median values')

Grouping similar titles together, I’ve kept a few titles - Officer, Royalty, Mr, Mrs and Miss.

train.imp <- train.raw
train.imp$title <- as.character(train.imp$title)
train.imp$title[train.imp$title %in% c('Capt','Col','Major')] <- 'Officer'
train.imp$title[train.imp$title %in% c('Don','Dr','Rev','Sir','Jonkheer','Countess','Lady','Dona')] <- 'Royalty'
train.imp$title[train.imp$title %in% c('Mrs','Mme')] <- 'Mrs'
train.imp$title[train.imp$title %in% c('Ms','Mlle')] <- 'Miss'
train.imp$title <- as.factor(train.imp$title)
train.imp %>%
    group_by(title) %>%
    summarise(Median_Age=median(Age,na.rm = T))
ggplot(train.imp,aes(x=title,y=Age))+
    geom_jitter(shape=21,alpha=.6,col='blue')+
    stat_summary(aes(y = Age,group=1), fun.y=median, colour="red", geom="point",group=1)+
    theme_bw()+
    theme(axis.text.x = element_text(angle = 45, hjust = 1),legend.position="none")+
    labs(caption='Red points are median values')

Now for the missing Age values. I’m trying out two strategies to impute age, just for kicks. First, a regression tree using the rpart method. 5-repeat 10-fold cross validation across a tuning grid of 20 values of maxdepth. RMSE stablizes at a depth of 14, with a value of 12.2.

age.predictors <- train.imp %>%
    dplyr::select(-Survived,-Cabin,-Ticket,-Name) %>%
    filter(complete.cases(.))
set.seed(1234)
ctrl <- trainControl(method = "boot",
                     repeats = 5,
                     number = 200
                     )
rpartGrid <- data.frame(maxdepth = seq(4,20,2))
rpartFit <- train(Age~.,
                  data=age.predictors,
                  method='rpart2',
                  trControl = ctrl,
                  tuneGrid = rpartGrid
                  )
rpartFit
CART 

712 samples
  7 predictor

No pre-processing
Resampling: Bootstrapped (200 reps) 
Summary of sample sizes: 712, 712, 712, 712, 712, 712, ... 
Resampling results across tuning parameters:

  maxdepth  RMSE      Rsquared 
   4        12.91352  0.2172555
   6        12.56362  0.2600303
   8        12.37466  0.2835666
  10        12.28184  0.2961068
  12        12.23967  0.3028092
  14        12.23329  0.3046570
  16        12.24043  0.3041673
  18        12.23669  0.3045630
  20        12.23821  0.3044234

RMSE was used to select the optimal model using  the smallest value.
The final value used for the model was maxdepth = 14.
plot(rpartFit)

plot(rpartFit$finalModel,margin=0.02)
text(rpartFit$finalModel,cex=0.8)

Another way is to run a randomforest with a search over values of mtry using 5-repeat 10-fold cross validation. As we can see mtry=4 is the optimal value which results in the lowest RMSE of 11.4; much better than the rpart model.

set.seed(1234)
rfGrid <- data.frame(mtry=seq(1,6,1))
ctrl <- trainControl(method = "repeatedcv",
                     repeats = 5
                     )
rfFit <- train(Age~.,
                  data=age.predictors,
                  method='rf',
                  trControl = ctrl,
                  tuneGrid = rfGrid)
rfFit
Random Forest 

712 samples
  7 predictor

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 642, 640, 642, 641, 640, 639, ... 
Resampling results across tuning parameters:

  mtry  RMSE      Rsquared 
  1     12.46449  0.3816241
  2     11.33714  0.4192503
  3     11.05166  0.4263448
  4     11.04766  0.4217797
  5     11.10717  0.4152716
  6     11.20324  0.4066238

RMSE was used to select the optimal model using  the smallest value.
The final value used for the model was mtry = 4.
plot(rfFit)

I’m going to use the randomForest model. Using the predict.train() to predict values of age and plug them back into the imputed data. You can see the blue points which are the imputed values of Age. What I noticed is that for all the titles, the imputed Age value seems to be distributed fairly well, except Master. For Master, the three imputed are definitely outliers. I’m going to force these to the median Age.

missing.age <- train.imp %>% filter(is.na(Age))
age.predicted <- predict(rfFit, newdata = missing.age)
train.imp %>%
    mutate(AgeMissing = is.na(Age),
           Age = ifelse(AgeMissing,age.predicted,Age)) %>%
    ggplot(aes(x=title,y=Age))+
    stat_summary(aes(y = Age,group=1), fun.y=median, colour="red", geom="point",group=1)+
    geom_jitter(aes(y=Age,col=AgeMissing),shape=2)+
    theme_bw()+
    theme(axis.text.x = element_text(angle = 45, hjust = 1),legend.position="none")+
    labs(caption='Red points are median values')

train.imp$Age[is.na(train.imp$Age)] <- age.predicted
train.imp$Age[train.imp$title=='Master' & train.imp$Age > 20] <- median(train.imp$Age[train.imp$title=='Master'],na.rm = T)

Derived Variables

Child?: Trying out two engineered variables here - is the passenger a child or not? Using Age=18 as a threshold. And is s/he close enough to be considered a adult by chance? Those between 16 and 18 could be mistaken for not being children. (My way of incorporating a fudge factor in the decision process of ladies & children first.)

train.imp$child <- 0
train.imp$child[train.imp$Age<18] <- 1
train.imp$almostadult <- as.numeric(between(train.imp$Age,16,18))

Really young, or really old?: Really young ones and older folks would get priority perhaps. Creating two categorical binary variables for these conditions.

train.imp$Young <- ifelse(train.imp$Age<10,1,0)
train.imp$Seniors <- ifelse(train.imp$Age>60,1,0)

Family related: Let’s also create some variables that talk about family sizes. What’s the total family size – continous variable TotalFam. Is the person single, part of a couple or a family? Three categorical variables for these.

train.imp$TotalFam <- train.imp$SibSp + train.imp$Parch + 1
# train.imp$LastName <- train.imp$Name %>% str_extract(pattern = '[a-zA-Z]+(?=,)')
# train.imp$FamSize <- paste0(train.imp$TotalFam,train.imp$LastName)
# train.imp$LastName <- NULL
train.imp$Name <- NULL
train.imp$LargeParCh <- as.numeric(train.imp$Parch>=3)
train.imp$LargeSibSp <- as.numeric(train.imp$SibSp>=3)
train.imp$Single <- ifelse(train.imp$TotalFam==1,1,0)
train.imp$Couple <- ifelse(train.imp$TotalFam==2,1,0)
train.imp$Family <- ifelse(train.imp$TotalFam>2,1,0)

Cabin related: Extracting the cabin alphabet and number from the cabin variable. Since the cabin numbers could be ordered from left to right or top to bottom on the boat, perhaps only the 1st digit is significant. Also, some folks have more than 1 cabin. Wonder if that’s important. Since lots of unknowns in the Cabin variable, all NA values are replaced by ‘U’. Refering to the deck diagram, the topmost decks are A and B, which are closest to the lifeboats. Perhaps that’s important too. Here, I create a bunch of categorical variables based off the original Cabin, and then remove it from the dataset.

train.imp$CabinMissing <- as.numeric(is.na(train.raw$Cabin))
train.imp$CabinCode <- map_chr(train.raw$Cabin,~str_split(string = .x,pattern = '')[[1]][1])
train.imp$CabinCode[is.na(train.imp$CabinCode)] <- 'U'
train.imp$CabinNum <- as.numeric(map_chr(train.raw$Cabin,~str_split(string = .x,pattern = '[a-zA-Z]')[[1]][2]))
train.imp$CabinNum <- map_int(train.imp$CabinNum, ~as.integer(str_split(.x,pattern = '',simplify = T)[1][1]))
train.imp$CabinNum[is.na(train.imp$CabinNum)] <- 0
train.imp$TopDeck <- ifelse(train.imp$CabinCode %in% c('A','B'),1,0)
train.imp$MidDeck <- ifelse(train.imp$CabinCode %in% c('C','D'),1,0)
train.imp$LowerDeck <- ifelse(train.imp$TopDeck==0 & train.imp$MidDeck ==0 ,1,0)
train.imp$NumberofCabins <- map_int(train.raw$Cabin,~str_split(string = .x,pattern = ' ')[[1]] %>% length)
train.imp$Cabin <- NULL

Ticket: Lastly, the ticket variable. I’m not sure what to make of it, so I’m keeping it for now, after cleaning it up a bit. A majority (80%) of the rows have unique (one) ticket. 14% rows have a duplicate ticket, perhaps indicating a family. A small number of rows have 3+ duplicates of the tickets.

train.imp$Ticket %>% table() %>% as.numeric() %>% table()
.
  1   2   3   4   5   6   7 
547  94  21  11   2   3   3 

There seems to be a bit of a pattern here. Tickets starting with 1 are mostly 1st class, those starting with 2 are 2nd class, and 3 - 3rd class. But, I feel it’s a very loose association.

train.imp %>% group_by(Pclass) %>% dplyr::select(Ticket,Pclass) %>% sample_n(5)

What I’m going to do is clean up the columns (remove special characters, spaces etc), then split the Ticket column into four: TicketChar, TicketNum,TicketNumLength, TicketNumStart. (Upon running the script a few times, I’ve decided to get rid of TicketNum, but I’m commenting the code for future ref). The TicketChar variable as this distribution:

train.imp %<>%
    mutate(
        Ticket = str_to_upper(Ticket) %>%
            str_replace_all(pattern = regex(pattern = '[.\\/]'),replacement = ''),
        TicketNum = str_extract(Ticket,pattern = regex('([0-9]){3,}')),
        TicketNumStart = map_int(TicketNum,~as.integer(str_split(.x,pattern = '',simplify = T)[1])),
        TicketNumLen = map_int(TicketNum,~dim(str_split(.x,pattern = '',simplify = T))[2]),
        TicketChar = str_extract(Ticket,pattern = regex('^[a-zA-Z/\\.]+')) 
        ) %>%
     mutate(
         TicketChar = map_chr(.x=TicketChar,
                              .f=~str_split(string=.x, pattern = '',simplify = T)[1])
         ) %>%     
    mutate(
        TicketChar = ifelse(is.na(TicketChar),'U',TicketChar),
        TicketNumStart = ifelse(is.na(TicketNumStart),0,TicketNumStart),
        TicketNumLen = ifelse(is.na(TicketNumLen),0,TicketNumLen),
    )
train.imp$Ticket <- NULL
train.imp$TicketNum <- NULL
table(train.imp$TicketChar)

  A   C   F   L   P   S   U   W 
 29  47   7   4  65  65 661  13 
table(train.imp$TicketNumLen)

  1   3   4   5   6   7 
  6   7 165 246 423  44 
table(train.imp$TicketNumStart)

  0   1   2   3   4   5   6   7   8   9 
  6 231 230 365  15   9  14  15   3   3 

Winzoring Variables

The fare variable has one massive outlier. Winzorising this variable using the 95th percentile value as the cutoff.

ggplot(train.imp,aes(x=Fare,fill=Pclass))+geom_histogram()+facet_grid(Pclass~.)

quantile(train.imp$Fare[train.imp$Pclass=='P1'],probs = c(.1,.25,.5,.75,.95))
      10%       25%       50%       75%       95% 
 26.55000  30.92395  60.28750  93.50000 232.52395 
train.imp$Fare[train.imp$Fare>232] <- 232

Final Data Review

The dataset is now prepared for modeling. Here’s a quick review of the data so far. 29 variables in total.

train.imp %>% glimpse()
Observations: 891
Variables: 29
$ Survived       <fctr> Dead, Survived, Survived, Survived, Dead, Dead, Dead, Dead, Survive...
$ Pclass         <fctr> P3, P1, P3, P1, P3, P3, P1, P3, P3, P2, P3, P1, P3, P3, P3, P2, P3,...
$ Sex            <fctr> male, female, female, female, male, male, male, male, female, femal...
$ Age            <dbl> 22.00000, 38.00000, 26.00000, 35.00000, 35.00000, 35.04936, 54.00000...
$ SibSp          <int> 1, 1, 0, 1, 0, 0, 0, 3, 0, 1, 1, 0, 0, 1, 0, 0, 4, 0, 1, 0, 0, 0, 0,...
$ Parch          <int> 0, 0, 0, 0, 0, 0, 0, 1, 2, 0, 1, 0, 0, 5, 0, 0, 1, 0, 0, 0, 0, 0, 0,...
$ Fare           <dbl> 7.2500, 71.2833, 7.9250, 53.1000, 8.0500, 8.4583, 51.8625, 21.0750, ...
$ Embarked       <fctr> S, C, S, S, S, Q, S, S, S, C, S, S, S, S, S, S, Q, S, S, C, S, S, Q...
$ title          <fctr> Mr, Mrs, Miss, Mrs, Mr, Mr, Mr, Master, Mrs, Mrs, Miss, Miss, Mr, M...
$ child          <dbl> 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1,...
$ almostadult    <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
$ Young          <dbl> 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,...
$ Seniors        <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
$ TotalFam       <dbl> 2, 2, 1, 2, 1, 1, 1, 5, 3, 2, 3, 1, 1, 7, 1, 1, 6, 1, 2, 1, 1, 1, 1,...
$ LargeParCh     <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
$ LargeSibSp     <dbl> 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0,...
$ Single         <dbl> 0, 0, 1, 0, 1, 1, 1, 0, 0, 0, 0, 1, 1, 0, 1, 1, 0, 1, 0, 1, 1, 1, 1,...
$ Couple         <dbl> 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,...
$ Family         <dbl> 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0,...
$ CabinMissing   <dbl> 1, 0, 1, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1,...
$ CabinCode      <chr> "U", "C", "U", "C", "U", "U", "E", "U", "U", "U", "G", "C", "U", "U"...
$ CabinNum       <dbl> 0, 8, 0, 1, 0, 0, 4, 0, 0, 0, 6, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 5, 0,...
$ TopDeck        <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,...
$ MidDeck        <dbl> 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0,...
$ LowerDeck      <dbl> 1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 1,...
$ NumberofCabins <int> 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,...
$ TicketNumStart <dbl> 2, 1, 3, 1, 3, 3, 1, 3, 3, 2, 9, 1, 2, 3, 3, 2, 3, 2, 3, 2, 2, 2, 3,...
$ TicketNumLen   <int> 5, 5, 7, 6, 6, 6, 5, 6, 6, 6, 4, 6, 4, 6, 6, 6, 6, 6, 6, 4, 6, 6, 6,...
$ TicketChar     <chr> "A", "P", "S", "U", "U", "U", "U", "U", "U", "U", "P", "U", "A", "U"...

Modeling

I’m experimenting with a few modeling techniques, mainly xgboost, gbm, and penalized models using glmnet. I’ve implemented all these models using caret which I find an absolutely indispensible toolkit to prep, build, tune and explore numerous models using very few lines of code.

For all models, I’m using a 5-repeat 10-fold cross validation technique on the training dataset. Thus, I have not split the training dataset further into test-train sets, given the small number of observations in the dataset.

Furthermore, given the 80:20 class-imbalance, I’m also trying out smote as an class balancing technique for a few models.

Tuning parameter searches (aka hypertuning) is performed using the tuneGrid parameter in the train() call. The best model is selected using the AUC of the ROC. Here are the models and a few intermediate results for each model. At the end, I’ve compared the performance of all the models together.

Extreme Gradient Boosting (xgboost)

xgbFit
eXtreme Gradient Boosting 

891 samples
 53 predictor
  2 classes: 'Survived', 'Dead' 

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 801, 803, 802, 802, 802, 801, ... 
Resampling results across tuning parameters:

  eta  max_depth  nrounds  ROC        Sens       Spec     
  0.3  2          2        0.8581505  0.6148067  0.9052660
  0.3  2          3        0.8677307  0.7174454  0.8699394
  0.3  2          4        0.8679025  0.7310084  0.8728485
  0.3  2          5        0.8704769  0.7262689  0.8768687
  0.3  2          6        0.8712057  0.7315630  0.8710303
  0.3  2          7        0.8718286  0.7338655  0.8706667
  0.3  3          2        0.8737598  0.7373782  0.8702963
  0.3  3          3        0.8771590  0.7403025  0.8750303
  0.3  3          4        0.8790200  0.7339160  0.8775623
  0.3  3          5        0.8804902  0.7333109  0.8815690
  0.3  3          6        0.8806026  0.7350420  0.8808620
  0.3  3          7        0.8807379  0.7256975  0.8856027
  0.3  4          2        0.8768955  0.7425882  0.8757710
  0.3  4          3        0.8805986  0.7373277  0.8794141
  0.3  4          4        0.8816917  0.7432101  0.8801145
  0.3  4          5        0.8826447  0.7444538  0.8819259
  0.3  4          6        0.8815038  0.7402185  0.8812121
  0.3  4          7        0.8804470  0.7273109  0.8833872
  0.3  5          2        0.8760521  0.7307059  0.8706397
  0.3  5          3        0.8774392  0.7290084  0.8706532
  0.3  5          4        0.8802163  0.7325210  0.8797845
  0.3  5          5        0.8793293  0.7395294  0.8775960
  0.3  5          6        0.8806162  0.7360336  0.8797778
  0.3  5          7        0.8828496  0.7377143  0.8797778
  0.3  6          2        0.8765941  0.7447059  0.8673535
  0.3  6          3        0.8772791  0.7359832  0.8706599
  0.3  6          4        0.8809166  0.7342857  0.8819057
  0.3  6          5        0.8833376  0.7430084  0.8804646
  0.3  6          6        0.8819891  0.7401008  0.8837508
  0.3  6          7        0.8839215  0.7388908  0.8844916
  0.3  7          2        0.8762051  0.7331092  0.8761010
  0.3  7          3        0.8775272  0.7423866  0.8812189
  0.3  7          4        0.8798993  0.7394286  0.8830370
  0.3  7          5        0.8803574  0.7435798  0.8852323
  0.3  7          6        0.8816307  0.7423866  0.8855960
  0.3  7          7        0.8818346  0.7429748  0.8918047
  0.5  2          2        0.8575588  0.6287899  0.8965320
  0.5  2          3        0.8669722  0.7303529  0.8714074
  0.5  2          4        0.8700492  0.7315126  0.8680943
  0.5  2          5        0.8704436  0.7315126  0.8721145
  0.5  2          6        0.8750264  0.7408571  0.8721145
  0.5  2          7        0.8764070  0.7454790  0.8786734
  0.5  3          2        0.8757504  0.7373950  0.8684714
  0.5  3          3        0.8780924  0.7397479  0.8695556
  0.5  3          4        0.8770632  0.7216134  0.8885320
  0.5  3          5        0.8781472  0.7327059  0.8819663
  0.5  3          6        0.8763049  0.7367227  0.8866734
  0.5  3          7        0.8769986  0.7308235  0.8874209
  0.5  4          2        0.8791924  0.7420000  0.8794074
  0.5  4          3        0.8805028  0.7425714  0.8739327
  0.5  4          4        0.8797489  0.7337983  0.8808889
  0.5  4          5        0.8792675  0.7314454  0.8856027
  0.5  4          6        0.8802721  0.7325042  0.8855960
  0.5  4          7        0.8808258  0.7366218  0.8866936
  0.5  5          2        0.8752125  0.7318319  0.8721010
  0.5  5          3        0.8778833  0.7303025  0.8775690
  0.5  5          4        0.8802324  0.7336807  0.8823030
  0.5  5          5        0.8792583  0.7296471  0.8819394
  0.5  5          6        0.8810007  0.7249412  0.8870438
  0.5  5          7        0.8802394  0.7249244  0.8844916
  0.5  6          2        0.8732269  0.7353613  0.8706465
  0.5  6          3        0.8761028  0.7377143  0.8801347
  0.5  6          4        0.8777128  0.7335462  0.8837845
  0.5  6          5        0.8791022  0.7341176  0.8841684
  0.5  6          6        0.8782975  0.7341176  0.8899731
  0.5  6          7        0.8771589  0.7323866  0.8892458
  0.5  7          2        0.8743506  0.7377815  0.8746330
  0.5  7          3        0.8772219  0.7419160  0.8874007
  0.5  7          4        0.8778936  0.7406723  0.8888687
  0.5  7          5        0.8804557  0.7441513  0.8903300
  0.5  7          6        0.8815295  0.7429580  0.8852189
  0.5  7          7        0.8810468  0.7377311  0.8877576

Tuning parameter 'gamma' was held constant at a value of 1
Tuning parameter
 was held constant at a value of 1
Tuning parameter 'subsample' was held constant at a value
 of 1
ROC was used to select the optimal model using  the largest value.
The final values used for the model were nrounds = 7, max_depth = 6, eta = 0.3, gamma =
 1, colsample_bytree = 1, min_child_weight = 1 and subsample = 1.
plot(xgbFit)
xgb.importance(feature_names = colnames(Dtrain),model = xgbFit$finalModel) %>%
    xgb.ggplot.importance()
densityplot(xgbFit,pch='|')
predict(xgbFit,type = 'prob') -> train.Probs
histogram(~Survived+Dead,train.Probs)

Extreme Gradient Boosting (xgboost) - SMOTE Sampling

ctrl <- trainControl(method = "repeatedcv",
                     repeats = 5,
                     verboseIter = T,
                     classProbs = TRUE,
                     summaryFunction = twoClassSummary,
                     sampling = 'smote'
                     )
xgbGrid <- expand.grid(
    nrounds=c(2,3,4,5,6,7),
    max_depth=c(2,3,4,5,6,7),
    eta=c(0.3,0.5),
    gamma=1,
    colsample_bytree=1,
    min_child_weight=1,
    subsample=1
)
dumV <- dummyVars(formula = Survived~.,data = train.imp)
Dtrain <- predict(dumV,train.imp)
variable 'Survived' is not a factor
xgbsmoteFit <- train(
    x=Dtrain,
    y=train.imp$Survived,
    method = 'xgbTree',
    trControl = ctrl,
    # metric = "Kappa",
    tuneGrid = xgbGrid,
    verbose = TRUE
)
The metric "Accuracy" was not in the result set. ROC will be used instead.
+ Fold01.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 

Attaching package: ‘DMwR’

The following object is masked from ‘package:plyr’:

    join
- Fold01.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep1: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep2: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep3: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep4: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold01.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold01.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold02.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold02.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold03.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold03.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold04.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold04.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold05.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold05.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold06.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold06.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold07.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold07.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold08.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold08.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold09.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold09.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.3, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.3, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.3, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.3, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.3, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.3, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.5, max_depth=2, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.5, max_depth=3, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.5, max_depth=4, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.5, max_depth=5, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.5, max_depth=6, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
+ Fold10.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
- Fold10.Rep5: eta=0.5, max_depth=7, gamma=1, colsample_bytree=1, min_child_weight=1, subsample=1, nrounds=7 
Aggregating results
Selecting tuning parameters
Fitting nrounds = 7, max_depth = 6, eta = 0.3, gamma = 1, colsample_bytree = 1, min_child_weight = 1, subsample = 1 on full training set
xgbsmoteFit
eXtreme Gradient Boosting 

891 samples
 53 predictor
  2 classes: 'Survived', 'Dead' 

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 802, 802, 802, 802, 802, 802, ... 
Addtional sampling using SMOTE

Resampling results across tuning parameters:

  eta  max_depth  nrounds  ROC        Sens       Spec     
  0.3  2          2        0.8422230  0.6017983  0.9041886
  0.3  2          3        0.8492899  0.6334454  0.9001886
  0.3  2          4        0.8559285  0.6701345  0.8968822
  0.3  2          5        0.8626094  0.6590252  0.9031044
  0.3  2          6        0.8654252  0.6521345  0.9060135
  0.3  2          7        0.8660548  0.6718824  0.9016296
  0.3  3          2        0.8594054  0.6228067  0.9235017
  0.3  3          3        0.8638679  0.5971429  0.9336768
  0.3  3          4        0.8659124  0.6034622  0.9326263
  0.3  3          5        0.8663275  0.6007563  0.9300404
  0.3  3          6        0.8692959  0.6066218  0.9307879
  0.3  3          7        0.8724934  0.6129412  0.9256700
  0.3  4          2        0.8552456  0.5870756  0.9348148
  0.3  4          3        0.8626897  0.5895462  0.9337104
  0.3  4          4        0.8694388  0.5895294  0.9358923
  0.3  4          5        0.8729766  0.5995126  0.9362559
  0.3  4          6        0.8755678  0.6064370  0.9366128
  0.3  4          7        0.8794687  0.6110252  0.9388081
  0.3  5          2        0.8600815  0.5929748  0.9329832
  0.3  5          3        0.8633800  0.5917983  0.9311515
  0.3  5          4        0.8689604  0.6000000  0.9347946
  0.3  5          5        0.8729573  0.6028739  0.9377104
  0.3  5          6        0.8751460  0.6105042  0.9340673
  0.3  5          7        0.8766301  0.6169580  0.9333199
  0.3  6          2        0.8633920  0.6239664  0.9191380
  0.3  6          3        0.8673741  0.6320504  0.9205859
  0.3  6          4        0.8715988  0.6268235  0.9231178
  0.3  6          5        0.8750872  0.6378992  0.9220471
  0.3  6          6        0.8761488  0.6419832  0.9209495
  0.3  6          7        0.8800014  0.6496134  0.9169630
  0.3  7          2        0.8561804  0.6192269  0.9195017
  0.3  7          3        0.8610940  0.6321008  0.9162222
  0.3  7          4        0.8646876  0.6397143  0.9176970
  0.3  7          5        0.8695185  0.6490924  0.9129428
  0.3  7          6        0.8710938  0.6572269  0.9143973
  0.3  7          7        0.8713630  0.6619496  0.9143906
  0.5  2          2        0.8490018  0.6370252  0.9012323
  0.5  2          3        0.8599325  0.6549748  0.9038047
  0.5  2          4        0.8629421  0.6410756  0.9125320
  0.5  2          5        0.8664901  0.6181849  0.9194815
  0.5  2          6        0.8674602  0.6101176  0.9202424
  0.5  2          7        0.8699278  0.6131933  0.9238451
  0.5  3          2        0.8610398  0.6136471  0.9242290
  0.5  3          3        0.8634444  0.5919160  0.9347744
  0.5  3          4        0.8695581  0.6042521  0.9296700
  0.5  3          5        0.8712536  0.5908571  0.9289495
  0.5  3          6        0.8726467  0.5941681  0.9347946
  0.5  3          7        0.8752367  0.6024370  0.9351515
  0.5  4          2        0.8626870  0.6024706  0.9329966
  0.5  4          3        0.8693973  0.5964034  0.9347879
  0.5  4          4        0.8755566  0.6059496  0.9315219
  0.5  4          5        0.8759294  0.6204874  0.9373401
  0.5  4          6        0.8739551  0.6158824  0.9373401
  0.5  4          7        0.8744622  0.6292269  0.9318855
  0.5  5          2        0.8631468  0.6064874  0.9278923
  0.5  5          3        0.8709386  0.6163361  0.9282357
  0.5  5          4        0.8742147  0.6315126  0.9267879
  0.5  5          5        0.8749953  0.6338992  0.9271582
  0.5  5          6        0.8747302  0.6426723  0.9238788
  0.5  5          7        0.8757683  0.6466891  0.9191515
  0.5  6          2        0.8625161  0.6239832  0.9213199
  0.5  6          3        0.8671210  0.6332605  0.9184108
  0.5  6          4        0.8676388  0.6419496  0.9111246
  0.5  6          5        0.8689798  0.6571765  0.9154949
  0.5  6          6        0.8728196  0.6530756  0.9125926
  0.5  6          7        0.8742729  0.6594958  0.9129495
  0.5  7          2        0.8649452  0.6513950  0.9213064
  0.5  7          3        0.8670273  0.6544034  0.9209562
  0.5  7          4        0.8748885  0.6684370  0.9198855
  0.5  7          5        0.8736510  0.6684034  0.9176835
  0.5  7          6        0.8763594  0.6748403  0.9180337
  0.5  7          7        0.8767229  0.6777983  0.9154949

Tuning parameter 'gamma' was held constant at a value of 1
Tuning parameter
 was held constant at a value of 1
Tuning parameter 'subsample' was held constant at a value
 of 1
ROC was used to select the optimal model using  the largest value.
The final values used for the model were nrounds = 7, max_depth = 6, eta = 0.3, gamma =
 1, colsample_bytree = 1, min_child_weight = 1 and subsample = 1.
plot(xgbsmoteFit)

xgb.importance(feature_names = colnames(Dtrain),model = xgbsmoteFit$finalModel)
xgb.importance(feature_names = colnames(Dtrain),model = xgbsmoteFit$finalModel) %>%
xgb.ggplot.importance()

densityplot(xgbsmoteFit,pch='|')

predict(xgbsmoteFit,type = 'raw') -> train.Class
predict(xgbsmoteFit,type = 'prob') -> train.Probs
histogram(~Survived+Dead,train.Probs)

Gradient Boosting (gbm)

boostFit
Stochastic Gradient Boosting 

891 samples
 53 predictor
  2 classes: 'Survived', 'Dead' 

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 803, 802, 802, 802, 802, 802, ... 
Resampling results across tuning parameters:

  shrinkage  interaction.depth  n.trees  ROC        Sens       Spec     
  0.01       1                   500     0.8687740  0.7176975  0.8812727
  0.01       1                   700     0.8715465  0.7310756  0.8827340
  0.01       1                   900     0.8731344  0.7456975  0.8798182
  0.01       1                  1100     0.8737417  0.7614454  0.8790909
  0.01       2                   500     0.8768379  0.7410252  0.8819933
  0.01       2                   700     0.8781806  0.7549916  0.8808956
  0.01       2                   900     0.8791223  0.7614118  0.8801751
  0.01       2                  1100     0.8801043  0.7613950  0.8809024
  0.01       3                   500     0.8790775  0.7526218  0.8823636
  0.01       3                   700     0.8808075  0.7584874  0.8812727
  0.01       3                   900     0.8815746  0.7549748  0.8809091
  0.01       3                  1100     0.8817104  0.7532269  0.8849091
  0.10       1                   500     0.8699484  0.7480168  0.8721751
  0.10       1                   700     0.8708636  0.7485882  0.8736162
  0.10       1                   900     0.8686151  0.7422017  0.8743367
  0.10       1                  1100     0.8689881  0.7428067  0.8725051
  0.10       2                   500     0.8801393  0.7473950  0.8837845
  0.10       2                   700     0.8799571  0.7467563  0.8819798
  0.10       2                   900     0.8790711  0.7438992  0.8790370
  0.10       2                  1100     0.8773468  0.7474118  0.8757778
  0.10       3                   500     0.8803987  0.7450252  0.8790774
  0.10       3                   700     0.8805280  0.7438655  0.8746869
  0.10       3                   900     0.8793306  0.7438655  0.8692189
  0.10       3                  1100     0.8785908  0.7432773  0.8713872

Tuning parameter 'n.minobsinnode' was held constant at a value of 10
ROC was used to select the optimal model using  the largest value.
The final values used for the model were n.trees = 1100, interaction.depth = 3, shrinkage
 = 0.01 and n.minobsinnode = 10.
plot(boostFit)

xyplot(oobag.improve~1:1100,data=boostFit$finalModel,alpha=.5,xlab = 'n.trees')

plot(varImp(boostFit))

densityplot(boostFit,pch='|')

predict(boostFit,type = 'prob') -> train.Probs
histogram(~Survived+Dead,train.Probs)

Random Forest (rf)

rfFit.y
Random Forest 

891 samples
 28 predictor
  2 classes: 'Survived', 'Dead' 

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 803, 802, 802, 802, 802, 802, ... 
Resampling results across tuning parameters:

  mtry  ROC        Sens       Spec     
   5    0.8842523  0.7122353  0.8969293
  10    0.8885360  0.7286891  0.8940202
  15    0.8874201  0.7351429  0.8878182
  20    0.8864149  0.7415462  0.8867138
  25    0.8866698  0.7427563  0.8816229

ROC was used to select the optimal model using  the largest value.
The final value used for the model was mtry = 10.
plot(rfFit.y)

plot(rfFit.y$finalModel)

densityplot(rfFit.y,pch='|')

predict(rfFit.y,type = 'prob') -> train.rf.Probs
histogram(~Survived+Dead,train.rf.Probs)

Random Forest (rf) - SMOTE

rfsmoteFit.y
Random Forest 

891 samples
 28 predictor
  2 classes: 'Survived', 'Dead' 

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 803, 802, 802, 802, 802, 802, ... 
Addtional sampling using SMOTE

Resampling results across tuning parameters:

  mtry  ROC        Sens       Spec     
   5    0.8783818  0.6333782  0.9369966
  10    0.8821756  0.6854286  0.9129697
  15    0.8831274  0.6983025  0.9016498
  20    0.8814043  0.7129580  0.8910976
  25    0.8793606  0.7135462  0.8885455

ROC was used to select the optimal model using  the largest value.
The final value used for the model was mtry = 15.
plot(rfsmoteFit.y)

plot(rfsmoteFit.y$finalModel)

densityplot(rfsmoteFit.y,pch='|')

predict(rfsmoteFit.y,type = 'prob') -> train.rfsmoteFit.Probs
histogram(~Survived+Dead,train.rfsmoteFit.Probs)

Elastinet

glmnetFit
glmnet 

891 samples
 53 predictor
  2 classes: 'Survived', 'Dead' 

No pre-processing
Resampling: Cross-Validated (10 fold, repeated 5 times) 
Summary of sample sizes: 803, 802, 802, 802, 802, 802, ... 
Addtional sampling using SMOTE

Resampling results across tuning parameters:

  alpha  lambda      ROC        Sens       Spec     
  0.0    0.01000000  0.8624351  0.7269580  0.8875152
  0.0    0.01487179  0.8624351  0.7269580  0.8875152
  0.0    0.01974359  0.8624351  0.7269580  0.8875152
  0.0    0.02461538  0.8624351  0.7269580  0.8875152
  0.0    0.02948718  0.8624351  0.7269580  0.8875152
  0.0    0.03435897  0.8626287  0.7269580  0.8871515
  0.0    0.03923077  0.8629192  0.7269412  0.8871515
  0.0    0.04410256  0.8631763  0.7269076  0.8860539
  0.0    0.04897436  0.8633421  0.7263361  0.8856902
  0.0    0.05384615  0.8631602  0.7246218  0.8860539
  0.0    0.05871795  0.8632236  0.7234286  0.8856902
  0.0    0.06358974  0.8634279  0.7240168  0.8853266
  0.0    0.06846154  0.8634157  0.7240168  0.8838721
  0.0    0.07333333  0.8634152  0.7246218  0.8835017
  0.0    0.07820513  0.8633267  0.7252269  0.8838653
  0.0    0.08307692  0.8633870  0.7246723  0.8842290
  0.0    0.08794872  0.8632672  0.7264370  0.8838653
  0.0    0.09282051  0.8630221  0.7258487  0.8838653
  0.0    0.09769231  0.8630112  0.7246723  0.8835017
  0.0    0.10256410  0.8630108  0.7258487  0.8827744
  0.0    0.10743590  0.8629793  0.7270252  0.8820337
  0.0    0.11230769  0.8629057  0.7275966  0.8820337
  0.0    0.11717949  0.8629160  0.7264370  0.8816700
  0.0    0.12205128  0.8629367  0.7252605  0.8816700
  0.0    0.12692308  0.8628505  0.7246723  0.8813064
  0.0    0.13179487  0.8627742  0.7252437  0.8813064
  0.0    0.13666667  0.8626347  0.7240840  0.8816700
  0.0    0.14153846  0.8626660  0.7234790  0.8816700
  0.0    0.14641026  0.8624861  0.7228908  0.8816700
  0.0    0.15128205  0.8623359  0.7205378  0.8809360
  0.0    0.15615385  0.8622288  0.7211261  0.8805724
  0.0    0.16102564  0.8621643  0.7211092  0.8798451
  0.0    0.16589744  0.8620677  0.7199496  0.8791111
  0.0    0.17076923  0.8620130  0.7193782  0.8787475
  0.0    0.17564103  0.8620221  0.7193782  0.8791111
  0.0    0.18051282  0.8620338  0.7193782  0.8791111
  0.0    0.18538462  0.8620142  0.7193782  0.8780202
  0.0    0.19025641  0.8620445  0.7182017  0.8765657
  0.0    0.19512821  0.8619043  0.7170252  0.8765657
  0.0    0.20000000  0.8618487  0.7152773  0.8758384
  0.1    0.01000000  0.8600372  0.7241176  0.8878519
  0.1    0.01487179  0.8615477  0.7264538  0.8896768
  0.1    0.01974359  0.8624891  0.7241345  0.8875017
  0.1    0.02461538  0.8631904  0.7223866  0.8878653
  0.1    0.02948718  0.8635738  0.7270252  0.8871246
  0.1    0.03435897  0.8640134  0.7281513  0.8867542
  0.1    0.03923077  0.8642901  0.7257983  0.8860269
  0.1    0.04410256  0.8644447  0.7257983  0.8852997
  0.1    0.04897436  0.8644893  0.7246218  0.8852997
  0.1    0.05384615  0.8647542  0.7234622  0.8845724
  0.1    0.05871795  0.8648624  0.7205378  0.8849360
  0.1    0.06358974  0.8646418  0.7205378  0.8849360
  0.1    0.06846154  0.8645368  0.7211261  0.8849360
  0.1    0.07333333  0.8643896  0.7193950  0.8842020
  0.1    0.07820513  0.8643594  0.7176303  0.8842020
  0.1    0.08307692  0.8642748  0.7141008  0.8845589
  0.1    0.08794872  0.8640778  0.7135294  0.8841953
  0.1    0.09282051  0.8636771  0.7123697  0.8834815
  0.1    0.09769231  0.8635362  0.7129580  0.8823906
  0.1    0.10256410  0.8631628  0.7123697  0.8805657
  0.1    0.10743590  0.8628222  0.7123529  0.8798316
  0.1    0.11230769  0.8625770  0.7106050  0.8794680
  0.1    0.11717949  0.8622052  0.7111933  0.8780135
  0.1    0.12205128  0.8619550  0.7111933  0.8772795
  0.1    0.12692308  0.8616472  0.7100336  0.8765522
  0.1    0.13179487  0.8616144  0.7088908  0.8761886
  0.1    0.13666667  0.8614748  0.7083025  0.8747340
  0.1    0.14153846  0.8612211  0.7082857  0.8747340
  0.1    0.14641026  0.8611024  0.7076975  0.8743636
  0.1    0.15128205  0.8609678  0.7065210  0.8740000
  0.1    0.15615385  0.8605709  0.7059328  0.8732727
  0.1    0.16102564  0.8604414  0.7059496  0.8725455
  0.1    0.16589744  0.8602385  0.7048067  0.8721751
  0.1    0.17076923  0.8599341  0.7036471  0.8725387
  0.1    0.17564103  0.8598033  0.7042353  0.8718114
  0.1    0.18051282  0.8595210  0.7036639  0.8707205
  0.1    0.18538462  0.8591656  0.7024874  0.8703569
  0.1    0.19025641  0.8589091  0.7018992  0.8699933
  0.1    0.19512821  0.8584617  0.7013277  0.8703569
  0.1    0.20000000  0.8581420  0.7013277  0.8692660
  0.2    0.01000000  0.8591220  0.7280672  0.8882020
  0.2    0.01487179  0.8604820  0.7245882  0.8903906
  0.2    0.01974359  0.8618651  0.7228571  0.8911178
  0.2    0.02461538  0.8620406  0.7228908  0.8892997
  0.2    0.02948718  0.8620432  0.7217143  0.8889360
  0.2    0.03435897  0.8622358  0.7234958  0.8889428
  0.2    0.03923077  0.8622652  0.7223361  0.8885859
  0.2    0.04410256  0.8620627  0.7229412  0.8889495
  0.2    0.04897436  0.8620116  0.7188235  0.8874949
  0.2    0.05384615  0.8619238  0.7153109  0.8864040
  0.2    0.05871795  0.8619083  0.7147227  0.8856700
  0.2    0.06358974  0.8614219  0.7141513  0.8853064
  0.2    0.06846154  0.8613769  0.7094790  0.8845724
  0.2    0.07333333  0.8613267  0.7089076  0.8849360
  0.2    0.07820513  0.8613301  0.7071429  0.8842088
  0.2    0.08307692  0.8613348  0.7065546  0.8823906
  0.2    0.08794872  0.8612440  0.7036471  0.8805724
  0.2    0.09282051  0.8607954  0.7036639  0.8791111
  0.2    0.09769231  0.8604293  0.7007563  0.8765657
  0.2    0.10256410  0.8599402  0.7007563  0.8747475
  0.2    0.10743590  0.8591161  0.7025378  0.8736431
  0.2    0.11230769  0.8580974  0.7019496  0.8721818
  0.2    0.11717949  0.8576971  0.7025042  0.8710774
  0.2    0.12205128  0.8571679  0.7025042  0.8703502
  0.2    0.12692308  0.8567174  0.7024874  0.8688956
  0.2    0.13179487  0.8561708  0.7007395  0.8656229
  0.2    0.13666667  0.8550908  0.7013277  0.8648956
  0.2    0.14153846  0.8544119  0.7007395  0.8645320
  0.2    0.14641026  0.8540218  0.7007563  0.8641684
  0.2    0.15128205  0.8534856  0.7007563  0.8638047
  0.2    0.15615385  0.8528506  0.7001681  0.8627138
  0.2    0.16102564  0.8523328  0.6984034  0.8627138
  0.2    0.16589744  0.8514521  0.6984034  0.8619865
  0.2    0.17076923  0.8508972  0.6978151  0.8598047
  0.2    0.17564103  0.8506594  0.6972269  0.8594411
  0.2    0.18051282  0.8500216  0.6978151  0.8583434
  0.2    0.18538462  0.8495585  0.6972437  0.8572458
  0.2    0.19025641  0.8490212  0.6966723  0.8561549
  0.2    0.19512821  0.8488122  0.6955126  0.8557912
  0.2    0.20000000  0.8477981  0.6949244  0.8547003
  0.4    0.01000000  0.8608629  0.7170924  0.8856835
  0.4    0.01487179  0.8630875  0.7188235  0.8875017
  0.4    0.01974359  0.8635522  0.7235126  0.8882290
  0.4    0.02461538  0.8638852  0.7241008  0.8863973
  0.4    0.02948718  0.8640961  0.7246387  0.8874882
  0.4    0.03435897  0.8636815  0.7205882  0.8878519
  0.4    0.03923077  0.8634645  0.7188403  0.8867542
  0.4    0.04410256  0.8625804  0.7165210  0.8845589
  0.4    0.04897436  0.8610777  0.7129916  0.8845589
  0.4    0.05384615  0.8603462  0.7106218  0.8812727
  0.4    0.05871795  0.8593766  0.7100168  0.8783569
  0.4    0.06358974  0.8586905  0.7106387  0.8758047
  0.4    0.06846154  0.8577189  0.7083193  0.8739865
  0.4    0.07333333  0.8561190  0.7071765  0.8707138
  0.4    0.07820513  0.8539432  0.7042521  0.8692525
  0.4    0.08307692  0.8520797  0.7048571  0.8674276
  0.4    0.08794872  0.8510941  0.7054454  0.8652391
  0.4    0.09282051  0.8498438  0.7060168  0.8612391
  0.4    0.09769231  0.8489821  0.7042521  0.8590572
  0.4    0.10256410  0.8485292  0.7054286  0.8543232
  0.4    0.10743590  0.8478570  0.7042521  0.8535960
  0.4    0.11230769  0.8474843  0.7013277  0.8514074
  0.4    0.11717949  0.8474952  0.7001345  0.8514141
  0.4    0.12205128  0.8472912  0.6995462  0.8503232
  0.4    0.12692308  0.8465502  0.6989580  0.8499596
  0.4    0.13179487  0.8458615  0.6977983  0.8495960
  0.4    0.13666667  0.8460465  0.6960336  0.8499596
  0.4    0.14153846  0.8456487  0.6936807  0.8510640
  0.4    0.14641026  0.8457106  0.6919328  0.8510640
  0.4    0.15128205  0.8463243  0.6884034  0.8514276
  0.4    0.15615385  0.8466792  0.6872269  0.8514276
  0.4    0.16102564  0.8467969  0.6860840  0.8514276
  0.4    0.16589744  0.8465443  0.6860840  0.8517912
  0.4    0.17076923  0.8460047  0.6854958  0.8521549
  0.4    0.17564103  0.8456889  0.6843193  0.8521549
  0.4    0.18051282  0.8452321  0.6843193  0.8521549
  0.4    0.18538462  0.8443937  0.6831765  0.8525185
  0.4    0.19025641  0.8439948  0.6831765  0.8525185
  0.4    0.19512821  0.8432182  0.6831765  0.8525185
  0.4    0.20000000  0.8429886  0.6820000  0.8525185
  0.6    0.01000000  0.8615146  0.7217311  0.8889428
  0.6    0.01487179  0.8625244  0.7234118  0.8911380
  0.6    0.01974359  0.8627385  0.7234622  0.8911246
  0.6    0.02461538  0.8630850  0.7211765  0.8900202
  0.6    0.02948718  0.8622451  0.7194286  0.8860000
  0.6    0.03435897  0.8620347  0.7135966  0.8830842
  0.6    0.03923077  0.8608737  0.7077311  0.8808956
  0.6    0.04410256  0.8595263  0.7077647  0.8776229
  0.6    0.04897436  0.8580028  0.7071765  0.8750774
  0.6    0.05384615  0.8563695  0.7054454  0.8725253
  0.6    0.05871795  0.8546962  0.7048739  0.8641616
  0.6    0.06358974  0.8528297  0.7066050  0.8601549
  0.6    0.06846154  0.8511820  0.7071765  0.8568822
  0.6    0.07333333  0.8498130  0.7089244  0.8521481
  0.6    0.07820513  0.8489291  0.7083361  0.8488687
  0.6    0.08307692  0.8492258  0.7071597  0.8488620
  0.6    0.08794872  0.8491236  0.7054118  0.8488620
  0.6    0.09282051  0.8492666  0.7018824  0.8488620
  0.6    0.09769231  0.8486469  0.7001176  0.8492256
  0.6    0.10256410  0.8485608  0.6954790  0.8506869
  0.6    0.10743590  0.8486213  0.6937143  0.8506869
  0.6    0.11230769  0.8478946  0.6931261  0.8506869
  0.6    0.11717949  0.8466370  0.6901849  0.8506869
  0.6    0.12205128  0.8461776  0.6860840  0.8517912
  0.6    0.12692308  0.8455451  0.6860840  0.8517912
  0.6    0.13179487  0.8436514  0.6854958  0.8517912
  0.6    0.13666667  0.8430642  0.6837647  0.8521549
  0.6    0.14153846  0.8423842  0.6831765  0.8521549
  0.6    0.14641026  0.8418510  0.6825882  0.8525185
  0.6    0.15128205  0.8407780  0.6820000  0.8525185
  0.6    0.15615385  0.8404541  0.6814118  0.8525185
  0.6    0.16102564  0.8401007  0.6814118  0.8525185
  0.6    0.16589744  0.8396466  0.6814118  0.8525185
  0.6    0.17076923  0.8391832  0.6814118  0.8525185
  0.6    0.17564103  0.8392284  0.6814118  0.8525185
  0.6    0.18051282  0.8390840  0.6814118  0.8525185
  0.6    0.18538462  0.8388220  0.6814118  0.8525185
  0.6    0.19025641  0.8389099  0.6814118  0.8525185
  0.6    0.19512821  0.8367532  0.6814118  0.8525185
  0.6    0.20000000  0.8350971  0.6814118  0.8525185
 [ reached getOption("max.print") -- omitted 80 rows ]

ROC was used to select the optimal model using  the largest value.
The final values used for the model were alpha = 0.1 and lambda = 0.05871795.
plot(glmnetFit,plotType='level')

plot(varImp(glmnetFit))

densityplot(glmnetFit,pch='|')

predict(glmnetFit,type = 'prob') -> train.glmnet.Probs
histogram(~Survived+Dead,train.glmnet.Probs)

Compare models

re <-
    resamples(
    x = list(
    xgb = xgbFit,
    xgbsmote = xgbsmoteFit,
    rf = rfFit.y,
    rfsmote = rfsmoteFit.y,
    gbm = boostFit,
    elastinet=glmnetFit
    )
    )
summary(re)

Call:
summary.resamples(object = re)

Models: xgb, xgbsmote, rf, rfsmote, gbm, elastinet 
Number of resamples: 50 

ROC 
               Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
xgb       0.7360963 0.8603304 0.8925134 0.8839215 0.9196505 0.9540260    0
xgbsmote  0.7462567 0.8653743 0.8867647 0.8800014 0.9078877 0.9350267    0
rf        0.7807487 0.8622670 0.8984301 0.8885360 0.9126833 0.9556150    0
rfsmote   0.7759358 0.8581818 0.8880026 0.8831274 0.9106551 0.9574866    0
gbm       0.7604278 0.8570834 0.8848587 0.8817104 0.9055828 0.9609626    0
elastinet 0.7518717 0.8413102 0.8650822 0.8648624 0.9010538 0.9636364    0

Sens 
               Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
xgb       0.5588235 0.7058824 0.7352941 0.7388908 0.7941176 0.9142857    0
xgbsmote  0.4411765 0.5882353 0.6470588 0.6496134 0.7058824 0.8529412    0
rf        0.5294118 0.6764706 0.7352941 0.7286891 0.7941176 0.8823529    0
rfsmote   0.5294118 0.6470588 0.6857143 0.6983025 0.7647059 0.9117647    0
gbm       0.5588235 0.6907563 0.7647059 0.7532269 0.8235294 0.9117647    0
elastinet 0.5294118 0.6470588 0.7390756 0.7205378 0.7714286 0.8529412    0

Spec 
               Min.   1st Qu.    Median      Mean   3rd Qu.      Max. NA's
xgb       0.7777778 0.8709596 0.8898990 0.8844916 0.9090909 0.9636364    0
xgbsmote  0.7818182 0.8909091 0.9272727 0.9169630 0.9454545 1.0000000    0
rf        0.8181818 0.8727273 0.8909091 0.8940202 0.9259259 0.9818182    0
rfsmote   0.8181818 0.8727273 0.8909091 0.9016498 0.9272727 0.9818182    0
gbm       0.8181818 0.8545455 0.8898990 0.8849091 0.9217172 0.9818182    0
elastinet 0.8000000 0.8545455 0.8909091 0.8849360 0.9090909 0.9636364    0
bwplot(re)

summary(diff(re))

Call:
summary.diff.resamples(object = diff(re))

p-value adjustment: bonferroni 
Upper diagonal: estimates of the difference
Lower diagonal: p-value for H0: difference = 0

ROC 
          xgb       xgbsmote   rf         rfsmote    gbm        elastinet 
xgb                  0.0039200 -0.0046145  0.0007941  0.0022111  0.0190591
xgbsmote  1.0000000            -0.0085345 -0.0031259 -0.0017090  0.0151390
rf        1.0000000 1.0000000              0.0054086  0.0068256  0.0236736
rfsmote   1.0000000 1.0000000  0.0511756              0.0014169  0.0182650
gbm       1.0000000 1.0000000  0.2458148  1.0000000              0.0168480
elastinet 0.7953846 1.0000000  2.206e-06  0.0007768  3.211e-05            

Sens 
          xgb       xgbsmote  rf        rfsmote   gbm       elastinet
xgb                  0.089277  0.010202  0.040588 -0.014336  0.018353
xgbsmote  0.0001186           -0.079076 -0.048689 -0.103613 -0.070924
rf        1.0000000 0.0001066            0.030387 -0.024538  0.008151
rfsmote   0.4046800 0.0365380 0.0001694           -0.054924 -0.022235
gbm       1.0000000 1.143e-06 0.0174450 9.138e-07            0.032689
elastinet 1.0000000 0.0036014 1.0000000 0.4417908 3.796e-05          

Spec 
          xgb      xgbsmote   rf         rfsmote    gbm        elastinet 
xgb                -3.247e-02 -9.529e-03 -1.716e-02 -4.175e-04 -4.444e-04
xgbsmote  0.014850             2.294e-02  1.531e-02  3.205e-02  3.203e-02
rf        1.000000 0.436352              -7.630e-03  9.111e-03  9.084e-03
rfsmote   0.503523 1.000000   0.229579               1.674e-02  1.671e-02
gbm       1.000000 0.027924   0.354081   0.007644              -2.694e-05
elastinet 1.000000 0.029234   1.000000   0.068587   1.000000             
simulatedTrain <- data.frame(Class = train.imp$Survived)
simulatedTrain$rf = predict(rfFit.y,type = 'prob')[[1]]
simulatedTrain$rfsmote = predict(rfsmoteFit.y,type = 'prob')[[1]]
simulatedTrain$xgb = predict(xgbFit,type = 'prob')[[1]]
simulatedTrain$xgbsmote = predict(xgbsmoteFit,type = 'prob')[[1]]
simulatedTrain$boost = predict(boostFit,type = 'prob')[[1]]
calCurve <- calibration(x = Class~rf+rfsmote+xgb+xgbsmote+boost,data = simulatedTrain)
xyplot(calCurve,auto.key=list(columns=3))

Calibrating probabilities

boostsigmoidCal <- glm(relevel(Class,ref='Dead')~boost,simulatedTrain,family = 'binomial')
coef(summary(boostsigmoidCal))
             Estimate Std. Error   z value     Pr(>|z|)
(Intercept) -3.393991  0.1992172 -17.03664 4.392812e-65
boost        7.063846  0.4064620  17.37886 1.193056e-67
simulatedTrain$boostSig = predict(boostsigmoidCal,type = 'response')
xgbsmotesigmoidCal <- glm(relevel(Class,ref='Dead')~xgbsmote,simulatedTrain,family = 'binomial')
coef(summary(xgbsmotesigmoidCal))
             Estimate Std. Error   z value     Pr(>|z|)
(Intercept) -3.468413  0.2016514 -17.20004 2.653386e-66
xgbsmote     9.205386  0.5945264  15.48356 4.479596e-54
simulatedTrain$xgbsmoteSig = predict(xgbsmotesigmoidCal,type = 'response')
calibration(x = Class~boost+boostSig,data = simulatedTrain) %>%
    xyplot(auto.key=list(columns=2))

calibration(x = Class~xgbsmote+xgbsmoteSig,data = simulatedTrain) %>%
    xyplot(auto.key=list(columns=2))

Evaluation

test.imp <- test.raw

#Embarked
test.imp$Embarked[is.na(test.imp$Embarked)]='S'

#Title
test.raw$title <- str_extract(pattern = '[a-zA-Z]+(?=\\.)',string = test.raw$Name)
#test.raw$title <- as.factor(test.raw$title)
test.imp$title <- as.character(test.raw$title)
test.imp$title[test.imp$title %in% c('Capt','Col','Major')] <- 'Officer'
test.imp$title[test.imp$title %in% c('Don','Dr','Rev','Sir','Jonkheer','Countess','Lady','Dona')] <- 'Royalty'
test.imp$title[test.imp$title %in% c('Mrs','Mme')] <- 'Mrs'
test.imp$title[test.imp$title %in% c('Ms','Mlle')] <- 'Miss'
test.imp$title <- as.factor(test.imp$title)

#Missing age
missing.age <- test.imp %>% filter(is.na(Age))
age.predicted <- predict(rfFit, newdata = missing.age)
test.imp$Age[is.na(test.imp$Age)] <- age.predicted
test.imp$Age[test.imp$title=='Master' & test.imp$Age > 20] <- 4

#Child
test.imp$child <- 0
test.imp$child[test.imp$Age<18] <- 1
test.imp$almostadult <- as.numeric(between(test.imp$Age,16,18))

#Young/old
test.imp$Young <- ifelse(test.imp$Age<10,1,0)
test.imp$Seniors <- ifelse(test.imp$Age>60,1,0)

#Family Related
test.imp$TotalFam <- test.imp$SibSp + test.imp$Parch + 1
test.imp$Name <- NULL
test.imp$LargeParCh <- as.numeric(test.imp$Parch>=3)
test.imp$LargeSibSp <- as.numeric(test.imp$SibSp>=3)
test.imp$Single <- ifelse(test.imp$TotalFam==1,1,0)
test.imp$Couple <- ifelse(test.imp$TotalFam==2,1,0)
test.imp$Family <- ifelse(test.imp$TotalFam>2,1,0)

#Cabin & Deck
test.imp$CabinMissing <- as.numeric(is.na(test.raw$Cabin))
test.imp$CabinCode <- map_chr(test.raw$Cabin,~str_split(string = .x,pattern = '')[[1]][1])
test.imp$CabinCode[is.na(test.imp$CabinCode)] <- 'U'
test.imp$CabinNum <- as.numeric(map_chr(test.raw$Cabin,~str_split(string = .x,pattern = '[a-zA-Z]')[[1]][2]))
test.imp$CabinNum <- map_int(test.imp$CabinNum, ~as.integer(str_split(.x,pattern = '',simplify = T)[1][1]))
test.imp$CabinNum[is.na(test.imp$CabinNum)] <- 0

test.imp$CabinCode <- factor(
    x = test.imp$CabinCode,
    levels = unique(train.imp$CabinCode)
)

test.imp$TopDeck <- ifelse(test.imp$CabinCode %in% c('A','B'),1,0)
test.imp$MidDeck <- ifelse(test.imp$CabinCode %in% c('C','D'),1,0)
test.imp$LowerDeck <- ifelse(test.imp$TopDeck==0 & test.imp$MidDeck ==0 ,1,0)

test.imp$NumberofCabins <- map_int(test.raw$Cabin,~str_split(string = .x,pattern = ' ')[[1]] %>% length)
test.imp$Cabin <- NULL


# Ticket
test.imp %<>%
    mutate(
      Ticket = str_to_upper(Ticket) %>%
          str_replace_all(pattern = regex(pattern = '[.\\/]'),replacement = ''),
      TicketNum = str_extract(Ticket,pattern = regex('([0-9]){3,}')),
      TicketNumStart = map_int(TicketNum,~as.integer(str_split(.x,pattern = '',simplify = T)[1])),
      TicketNumLen = map_int(TicketNum,~dim(str_split(.x,pattern = '',simplify = T))[2]),
      TicketChar = str_extract(Ticket,pattern = regex('^[a-zA-Z/\\.]+'))
      ) %>%
    mutate(
        TicketChar = map_chr(.x=TicketChar,
                             .f=~str_split(string=.x, pattern = '',simplify = T)[1])
        ) %>%  
    mutate(
      TicketChar = ifelse(is.na(TicketChar),'U',TicketChar),
      TicketNumStart = ifelse(is.na(TicketNumStart),0,TicketNumStart),
      TicketNumLen = ifelse(is.na(TicketNumLen),0,TicketNumLen),
    )
test.imp$Ticket <- NULL
test.imp$TicketNum <- NULL

#Fare
test.imp$Fare[is.na(test.imp$Fare)] <- 14.4542
test.imp$Fare[test.imp$Fare>232] <- 232
xgbsmotesigmoidCal

Call:  glm(formula = relevel(Class, ref = "Dead") ~ xgbsmote, family = "binomial", 
    data = simulatedTrain)

Coefficients:
(Intercept)     xgbsmote  
     -3.468        9.205  

Degrees of Freedom: 890 Total (i.e. Null);  889 Residual
Null Deviance:      1187 
Residual Deviance: 491.9    AIC: 495.9
PID <-
    readData(Titanic.path,
    test.data.file,
    test.column.types,
    missing.types)
PID <- PID$PassengerId
for(m in ls(pattern = 'Pred')) {
    write.csv(
    x = data.frame(
    PassengerId = PID,
    Survived = as.numeric(eval(parse(text = m))) * -1 + 2
    ),
    file = paste0(m,'.csv'),
    row.names = F
    )
}

Conclusions


  1. I think this approach depends on the academic background and the industry of the analyst. Prof Srinivasan, and my mentor at work both have strong statistical academic backgrounds, and both believe in thorough EDA of the data. I’ve also noticed this approach from individuals in the banking & insurance industry - perhaps due to regulatory requirements. On the other hand, folks trained in computer science and algorithmic data science tend to underplay the importance of thorough EDA.

  2. To iterate variable names in ggplot, use ggplot(...)+aes_string(...) in place of ggplot(...,aes(...)).

  3. Read more about beanplots here: https://cran.r-project.org/web/packages/beanplot/vignettes/beanplot.pdf

LS0tCnRpdGxlOiAiS2FnZ2xlIFRpdGFuaWMgQ29tcGV0aXRpb24gLSBEYXRhIEV4cGxvcmF0aW9uICYgTW9kZWwgQnVpbGRpbmciCmF1dGhvcjogJ1JhaHVsIFNhbmdvbGUnCmRhdGU6IEF1ZyAzMSwgMjAxNwpvdXRwdXQ6CiAgaHRtbF9ub3RlYm9vazoKICAgIGNvZGVfZm9sZGluZzogaGlkZQogICAgY29sbGFwc2VkOiBubwogICAgZmlnX2hlaWdodDogMwogICAgZmlnX3dpZHRoOiA1CiAgICBoaWdobGlnaHQ6IHB5Z21lbnRzCiAgICB0aGVtZTogam91cm5hbAogICAgdG9jOiB5ZXMKICAgIHRvY19mbG9hdDogeWVzCi0tLQojIE9iamVjdGl2ZXMKCjEuIEVuZCB0byBlbmQgYW5hbHlzaXMgdXNpbmcgUgoyLiBMZWFybiB0aGUgY2FyZXQgcGFja2FnZSBmb3IgTUwKMy4gTGVhcm4gdG8gcHJlc2VudCB0aGUgY2FzZSB1c2luZyBSIE5vdGVib29rcwoKKioqCgojIFJlYWQgaW4gdGhlIGRhdGFzZXQKSSBzdG9yZWQgdGhlIHJhdyBmaWxlcyBvbiBHaXRodWIsIHNvIEkgdXNlZCBbUkN1cmxdKGh0dHBzOi8vY3Jhbi5yLXByb2plY3Qub3JnL3dlYi9wYWNrYWdlcy9SQ3VybC9pbmRleC5odG1sKSB3aXRoIFtXZWhybGV5J3MgbWV0aG9kXShodHRwczovL2dpdGh1Yi5jb20vd2VocmxleS93ZWhybGV5LmdpdGh1Yi5pby9ibG9iL21hc3Rlci9TT1VQVE9OVVRTLm1kKSB0aGF0IHV0aWxpemVzIHJlYWQuY3N2IHRvIHRoZSBmdWxsZXN0LiBJdCdzIG9uZSBvZiB0aGUgYmVzdCB3YXlzIEkndmUgZm91bmQgdG8gcmVhZCBpbiBkYXRhIGFuZCBhbHNvIHNldCBkYXRhLXR5cGVzIGF0IHRoZSBzYW1lIHRpbWUuIEhlJ3MgZG9uZSBhIGdyZWF0IGpvYiBvbiB0aGF0IGZ1bmN0aW9uLiBUaGUgZGF0YXNldCBjb250YWlucyBvbmUgSUQgdmFyaWFibGUsIG9uZSByZXNwb25zZSB2YXJpYWJsZSBhbmQgdGVuIHByZWRpY3RvciB2YXJpYWJsZXMuCgpgYGB7ciwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KbGlicmFyeShSQ3VybCxxdWlldGx5ID0gVCkKbGlicmFyeSh0aWR5dmVyc2UscXVpZXRseSA9IFQpCmxpYnJhcnkoZ2dwbG90MixxdWlldGx5ID0gVCkKbGlicmFyeShncmlkRXh0cmEscXVpZXRseSA9IFQpCmxpYnJhcnkoQW1lbGlhLHF1aWV0bHkgPSBUKQpsaWJyYXJ5KGJlYW5wbG90LHF1aWV0bHkgPSBUKQpsaWJyYXJ5KGNhcmV0LHF1aWV0bHkgPSBUKQpsaWJyYXJ5KHN0cmluZ3IscXVpZXRseSA9IFQpCmxpYnJhcnkocGFydHksIHF1aWV0bHkgPSBUKQojIGxpYnJhcnkocmF0dGxlLCBxdWlldGx5ID0gVCkKCnJlYWREYXRhIDwtIGZ1bmN0aW9uKHBhdGgubmFtZSwgZmlsZS5uYW1lLCBjb2x1bW4udHlwZXMsIG1pc3NpbmcudHlwZXMpIHsKICAgIGd1cmwgPC0gcGFzdGUocGF0aC5uYW1lLGZpbGUubmFtZSxzZXA9IiIpCiAgICBkb3dubG9hZC5maWxlKGd1cmwsZmlsZS5uYW1lLG1ldGhvZD0iY3VybCIscXVpZXQgPSBUKQogICAgdGJsX2RmKHJlYWQuY3N2KGZpbGUubmFtZSxjb2xDbGFzc2VzPWNvbHVtbi50eXBlcywKICAgICAgICAgICAgIG5hLnN0cmluZ3M9bWlzc2luZy50eXBlcykpCn0KClRpdGFuaWMucGF0aCA8LSAiaHR0cHM6Ly9yYXcuZ2l0aHVidXNlcmNvbnRlbnQuY29tL3JzYW5nb2xlL1RpdGFuaWMvbWFzdGVyLyIKdHJhaW4uZGF0YS5maWxlIDwtICJ0cmFpbi5jc3YiCnRlc3QuZGF0YS5maWxlIDwtICJ0ZXN0LmNzdiIKbWlzc2luZy50eXBlcyA8LSBjKCJOQSIsICIiKQp0cmFpbi5jb2x1bW4udHlwZXMgPC0gYygnaW50ZWdlcicsICAgIyBQYXNzZW5nZXJJZAogICAgICAgICAgICAgICAgICAgICAgICAnZmFjdG9yJywgICAgIyBTdXJ2aXZlZAogICAgICAgICAgICAgICAgICAgICAgICAnZmFjdG9yJywgICAgIyBQY2xhc3MKICAgICAgICAgICAgICAgICAgICAgICAgJ2NoYXJhY3RlcicsICMgTmFtZQogICAgICAgICAgICAgICAgICAgICAgICAnZmFjdG9yJywgICAgIyBTZXgKICAgICAgICAgICAgICAgICAgICAgICAgJ251bWVyaWMnLCAgICMgQWdlCiAgICAgICAgICAgICAgICAgICAgICAgICdpbnRlZ2VyJywgICAjIFNpYlNwCiAgICAgICAgICAgICAgICAgICAgICAgICdpbnRlZ2VyJywgICAjIFBhcmNoCiAgICAgICAgICAgICAgICAgICAgICAgICdjaGFyYWN0ZXInLCAjIFRpY2tldAogICAgICAgICAgICAgICAgICAgICAgICAnbnVtZXJpYycsICAgIyBGYXJlCiAgICAgICAgICAgICAgICAgICAgICAgICdjaGFyYWN0ZXInLCAjIENhYmluCiAgICAgICAgICAgICAgICAgICAgICAgICdmYWN0b3InICAgICAjIEVtYmFya2VkCikKCnRlc3QuY29sdW1uLnR5cGVzIDwtIHRyYWluLmNvbHVtbi50eXBlc1stMl0gICAgICMgIyBubyBTdXJ2aXZlZCBjb2x1bW4gaW4gdGVzdC5jc3YKdHJhaW4ucmF3IDwtIHJlYWREYXRhKFRpdGFuaWMucGF0aCwgdHJhaW4uZGF0YS5maWxlLHRyYWluLmNvbHVtbi50eXBlcyxtaXNzaW5nLnR5cGVzKQp0ZXN0LnJhdyA8LSByZWFkRGF0YShUaXRhbmljLnBhdGgsIHRlc3QuZGF0YS5maWxlLHRlc3QuY29sdW1uLnR5cGVzLG1pc3NpbmcudHlwZXMpCgpwcmVwX2RhdGEgPC0gZnVuY3Rpb24oRCkgewogICAgaWYgKCFpcy5udWxsKEQkU3Vydml2ZWQpKSB7CiAgICAgICAgRCRTdXJ2aXZlZCA8LSBmYWN0b3IoRCRTdXJ2aXZlZCwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBsZXZlbHMgPSBjKDEsIDApLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgIGxhYmVscyA9IGMoJ1N1cnZpdmVkJywgJ0RlYWQnKSkKICAgICAgICB9CiAgICBEJFBjbGFzcyA8LSBmYWN0b3IoRCRQY2xhc3MsCiAgICAgICAgICAgICAgICAgICAgICAgbGV2ZWxzID0gYygxLCAyLCAzKSwKICAgICAgICAgICAgICAgICAgICAgICBsYWJlbHMgPSBjKCdQMScsICdQMicsICdQMycpKQogICAgRCRQYXNzZW5nZXJJZCA8LSBOVUxMCiAgICBECn0KCnRyYWluLnJhdyA8LSBwcmVwX2RhdGEodHJhaW4ucmF3KQp0ZXN0LnJhdyA8LSBwcmVwX2RhdGEodGVzdC5yYXcpCnN0cih0cmFpbi5yYXcpCmBgYAoKKioqCgojIE1pc3NpbmcgdmFsdWVzIGFuYWx5c2lzCgpRdWljayBpbnZlc3RpZ2F0aW9uIG9mIG1pc3NpbmcgdmFsdWVzIGNhbiBiZSBkb25lIHVzaW5nIHRoZSBgY29tcGxldGUuY2FzZXMoKWAsIGFuZCBtb3JlIHRob3JvdWdoIGdyYXBoaWNhbCBzdW1tYXJ5IGNhbiBiZSBkb25lIHVzaW5nIEFtZWxpYS4gT3ZlcmFsbCwgNzklIG9mIHRoZSBvYnNlcnZhdGlvbnMgaGF2ZSAqc29tZSogbWlzc2luZyBkYXRhLgoKYGBge3J9CiNDb21wbGV0ZSBjYXNlcyAocGVyY2VudGFnZXMpCnJvdW5kKHByb3AudGFibGUodGFibGUoY29tcGxldGUuY2FzZXModHJhaW4ucmF3KSkpLDIpCmBgYApBbWVsaWEgbGV0cyB1cyBncmFwaGljYWxseSBpbnZlc3RpZ2F0ZSB3aGljaCB2YXJpYWJsZXMgaGF2ZSBtaXNzaW5nIGRhdGEuIGBwdXJyOjptYXBfeHh4KClgIGdpdmVzIHRoaXMgc2FtZSBpbmZvcm1hdGlvbiBudW1lcmljYWxseSBpbiBhIHN1Y2NpbnQgZmFzaGlvbi4KYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9Cm1pc3NtYXAodHJhaW4ucmF3LCBtYWluPSdNaXNzaW5nIFZhbHVlcyBBbmFseXNpcyB1c2luZyBBbWVsaWEgb3JkZXJlZCBieSAlIG1pc3NpbmcnLCBjb2w9YygncmVkJywgJ2dyYXknKSxsZWdlbmQgPSBGLHJhbmsub3JkZXIgPSBUKQojTWlzc2luZyBjYXNlcyAobnVtYmVycyk6Cm1hcF9pbnQodHJhaW4ucmF3LH5zdW0oaXMubmEoLngpKSkKI01pc3NpbmcgY2FzZXMgKHBlcmNlbnRhZ2VzKToKcm91bmQobWFwX2RibCh0cmFpbi5yYXcsfnN1bShpcy5uYSgueCkpL2xlbmd0aCgueCkpLDIpCmBgYApDYWJpbiBoYXMgYSBsYXJnZSBudW1iZXIgb2YgbWlzc2luZyB2YWx1ZXMgKDc3JSBtaXNzaW5nKS4gSW1wdXRpbmcgdGhpcyB2YXJpYWJsZSBtYXkgcHJvdmUgY2hhbGxlbmdpbmcgb3IgZXZlbiB1c2VsZXNzLiBBZ2UgKDE5LjklIG1pc3NpbmcpIGFuZCBFbWJhcmtlZCAoMC4yJSkgbWlzc2luZyBhcmUgbXVjaCBtb3JlIG1hbmFnYWJsZS4KCioqKgoKIyBFREEKClRoZSBmaXJzdCBzdGVwIGluIHRoZSBhbmFseXNpcyBpcyB0byBleHBsb3JlIHRoZSBkYXRhIG51bWVyaWNhbGx5IGFuZCBncmFwaGljYWxseS4gSSBhbHdheXMgc3BsaXQgdXAgbXkgRURBIGludmVzdGlnYXRpb24gYXMgZm9sbG93czoKCiogVGFyZ2V0IFZhcmlhYmxlCiogUHJlZGljdG9yIFZhcmlhYmxlcwogICAgKyBVbml2YXJpYXRlCiAgICArIEJpdmFyaWF0ZQogICAgKyBNdWx0aXZhcmlhdGUKClRoaXMgZ2l2ZXMgbWUgYSBzdHJ1Y3R1cmVkIGFwcHJvYWNoIHRvd2FyZHMgbGFyZ2VyIGRhdGFzZXRzLiBNeSBbcHJvZmVzc29yXShodHRwOi8vd3d3LnN5YW1hbGFzcmluaXZhc2FuLmNvbS8pIGF0IE5vcnRod2VzdGVybiB0YXVnaHQgbWUgdG8gYWx3YXlzIGNvbXBsZXRlIGEgdGhvcm91Z2ggaW50aW1hdGUgbnVtZXJpYyAmIGdyYXBoaWNhbCBFREEgb24gdGhlIGRhdGEsIG5vIG1hdHRlciBob3cgbGFyZ2UgdGhlIGRhdGEgW14xXS4gW0Fuc2NvbWJlXShodHRwOi8vd3d3LmpzdG9yLm9yZy9zdGFibGUvMjY4Mjg5OSkgKDE5NzMpIGNsZWFybHkgc2hvd3MgdGhlIGltcG9ydGFuY2Ugb2YgZ3JhcGhpY2FsIGFuYWx5c2VzLgoKW14xXTogSSB0aGluayB0aGlzIGFwcHJvYWNoIGRlcGVuZHMgb24gdGhlIGFjYWRlbWljIGJhY2tncm91bmQgYW5kIHRoZSBpbmR1c3RyeSBvZiB0aGUgYW5hbHlzdC4gUHJvZiBTcmluaXZhc2FuLCBhbmQgbXkgbWVudG9yIGF0IHdvcmsgYm90aCBoYXZlIHN0cm9uZyBzdGF0aXN0aWNhbCBhY2FkZW1pYyBiYWNrZ3JvdW5kcywgYW5kIGJvdGggYmVsaWV2ZSBpbiB0aG9yb3VnaCBFREEgb2YgdGhlIGRhdGEuIEkndmUgYWxzbyBub3RpY2VkIHRoaXMgYXBwcm9hY2ggZnJvbSBpbmRpdmlkdWFscyBpbiB0aGUgYmFua2luZyAmIGluc3VyYW5jZSBpbmR1c3RyeSAtIHBlcmhhcHMgZHVlIHRvIHJlZ3VsYXRvcnkgcmVxdWlyZW1lbnRzLiBPbiB0aGUgb3RoZXIgaGFuZCwgZm9sa3MgdHJhaW5lZCBpbiBjb21wdXRlciBzY2llbmNlIGFuZCBhbGdvcml0aG1pYyBkYXRhIHNjaWVuY2UgdGVuZCB0byB1bmRlcnBsYXkgdGhlIGltcG9ydGFuY2Ugb2YgdGhvcm91Z2ggRURBLgoKIyMgVGFyZ2V0IFZhcmlhYmxlCmBTdXJ2aXZlZGAgaXMgdGhlIHJlc3BvbnNlIHZhcmlhYmxlLiBBcyB3ZSBjYW4gc2VlLCBhIGxhcmdlIG1ham9yaXR5IG9mIHRoZSBwYXNzZW5nZXJzIGRpZCBub3Qgc3Vydml2ZSB0aGUgYWNjaWRlbnQuIFRoZSByZXNwb25zZSB2YXJpYWJsZSBpcyBhIEZhbHNlL1RydWUgYm9vbGVhbiB2YXJpYWJsZS4gVGh1cywgdGhlIGFuYWx5c2lzIHRlY2huaXF1ZXMgdXNlZCBsYXRlciB3aWxsIGJlIHRob3NlIGFwcHJvcHJpYXRlIGZvciBjbGFzc2lmaWNhdGlvbiBwcm9ibGVtcy4KYGBge3J9CnJvdW5kKHByb3AudGFibGUodGFibGUodHJhaW4ucmF3JFN1cnZpdmVkKSksMikKYGBgCgoqKioKCiMjIFByZWRpY3RvciBWYXJpYWJsZXMgey50YWJzZXQgLnRhYnNldC1mYWRlfQoKIyMjIFVuaXZhcmlhdGUgJiBCaXZhcmlhdGUKClRoZSBmaXJzdCBzdGVwIGlzIHRvIGxvb2sgYXQgZXZlcnkgdmFyaWFibGUgYXZhaWxhYmxlLiBJIHByZWZlciB1c2luZyB0aGUgYGdncGxvdDJgIGZyYW1ld29yayBmb3IgYWxsIHRoZSB2aXN1YWxzLgoKIyMjIyBDb250aW51b3VzIFZhcmlhYmxlcwoKKiBgQWdlYCBzZWVtcyB0byBoYXZlIGEgYmltb2RhbCBkaXN0cmlidXRpb24gLSB2ZXJ5IHlvdW5nIGNoaWxkcmVuLCBhbmQgdGhlbiBkaXJlY3RseSB5b3VuZyBhZHVsdHMgdG8gbWlkLWFnZSBwZXJzb25zLiBUaGUgMm5kIG1vZGUgaXMgcmlnaHQgc2tld2VkIHdpdGggbm8gb2J2aW91cyBvdXRsaWVycy4KCiogYEZhcmVgIGNlcnRhaW5seSBzaG93cyBtYW55IG91dGxpZXJzIGJleW9uZCB0aGUgfiQyMDAgbGV2ZWwuIEEgbWFqb3JpdHkgb2YgdGhlIGZhcmVzIGFyZSA8JDUwLCB3aGljaCBtYWtlcyBzZW5zZSBzaW5jZSBhIG1ham9yaXR5IG9mIHRoZSB0cmF2ZWxlcnMgYXJlIGJvdW5kIHRvIGJlIGluIHRoZSAzcmQgcGFzc2VuZ2VyIGNsYXNzLgoKYGBge3IsIG1lc3NhZ2U9RkFMU0UsIHdhcm5pbmc9RkFMU0V9CnAxIDwtIGdncGxvdChkYXRhPXRyYWluLnJhdyxhZXMoeD1BZ2UpKStnZW9tX2hpc3RvZ3JhbShiaW5zID0gNDApCnAyIDwtIGdncGxvdChkYXRhPXRyYWluLnJhdyxhZXMoeD1GYXJlKSkrZ2VvbV9oaXN0b2dyYW0oYmlucyA9IDQwKQpncmlkLmFycmFuZ2UocDEscDIpCmBgYAoKQXMgd2UgY2FuIHNlZSwgdGhlIG1lZGlhbiBmYXJlIGlzICQxNC41LCB0aGUgbWVhbiBpcyAkMzIsIGJ1dCB0aGUgbWF4IGlzICQ1MTIuIFdlJ2xsIGludmVzdGlnYXRlIHdpbnpvcmlzaW5nIHRoaXMgdmFyaWFibGUgaW4gdGhlIGxhdHRlciBwYXJ0LiBQZXJoYXBzIGEgdHJhbnNmb3JtYXRpb24gd2lsbCBhbHNvIGhlbHA/CgpgYGB7cn0Kc3VtbWFyeSh0cmFpbi5yYXckRmFyZSkKYGBgCgojIyMjIENhdGVnb3JpY2FsIFZhcmlhYmxlcwoKQSBnZ3Bsb3QgY29tbWFuZCBpcyBpdGVyYXRlZCBvdmVyIGZvciB0aGUgY2F0ZWdvcmljYWwgdmFyaWFibGVzLlteMl0KClteMl06IFRvIGl0ZXJhdGUgdmFyaWFibGUgbmFtZXMgaW4gZ2dwbG90LCB1c2UgYGdncGxvdCguLi4pK2Flc19zdHJpbmcoLi4uKWAgaW4gcGxhY2Ugb2YgYGdncGxvdCguLi4sYWVzKC4uLikpYC4KCktleSB0YWtld2F5cyBmb3IgdGhlIGNhdGVnb3JpY2FsIHZhcmlhYmxlczoKCjEuIGBQY2xhc3NgOiBJZiB5b3Ugd2VyZSB0cmF2ZWxpbmcgMXN0IGNsYXNzLCB5b3UgaGF2ZSB0aGUgaGlnaGVzdCBjaGFuY2Ugb2Ygc3Vydml2YWwuIENvdWxkIGJlIGluZGljYXRpdmUgb2YgcHJlZmVyZW50aWFsIHRyZWF0bWVudCB0byB0aG9zZSB3aG8gcGFpZCBtb3JlLCBhIGxlc3MgcG9saXRpY2FsbHkgY29ycmVjdCBjbGFzcy1zdHJhdGlmaWVkIHNvY2lldHksIGFzIHdlbGwgYXMgdGhlIGZhY3QgdGhhdCB0aGUgMXN0IGNsYXNzIHBhc3NlbmdlcnMgaGFkIGNhYmlucyBhdCB0aGUgdmVyeSB0b3Agb2YgdGhlIHNoaXAuCjIuIGBQY2xhc3NgOiBQZXJzb25zIHRyYXZlbGluZyAzcmQgY2xhc3MgaGFkIHRoZSBoaWdoZXN0IGZhdGFsaXR5IHJhdGUuIDNyZCBjbGFzcyBwYXNzZW5nZXJzIGhhZCBjYWJpbnMgZGVlcCBpbiB0aGUgc2hpcC4gV2l0aCB0aGUgcmVhc29ucyBnaXZlIGluICgxKSwgdGhpcyBjb3VsZCBoYXZlIGNvbnRyaWJ1dGVkIHRvIHRoZSBsb3cgc3Vydml2YWwgcmF0ZS4KMy4gYFNleGA6IE1hbGVzIGhhdmUgYSB2ZXJ5IGhpZ2ggZmF0YWxpdHkgcmF0ZS4gU2VlbXMgbGlrZSB0aGUgJ3dvbWVuIGFuZCBjaGlsZHJlbicgZmlyc3QgcG9saWN5IHdhcyBmb2xsb3dlZCBkdXJpbmcgZXZhY3VhdGlvbi4KNC4gYFNpYlNwYCAmIGBQYXJjaGA6IFdoYXQncyBpbnRlcmVzdGluZyBoZXJlIGlzLCBmb3IgYm90aCB0aGVzZSB2YXJpYWJsZXMsIGF0IGxldmVsIDAsIHRoZSBmYXRhbGl0eSByYXRlIGlzIGhpZ2hlci4gQXQgbGV2ZWxzIDErLCB0aGUgY2hhbmNlcyBvZiBzdXJ2aXZhbCBhcmUgbXVjaCBiZXR0ZXIuIEFnYWluLCB0aGlzIGNvdWxkIHBvaW50IHRvIHRoZSAnd29tZW4gKmFuZCBjaGlsZHJlbionIHBvbGljeSBiZWluZyBmb2xsb3dlZC4gKE9yIHBlcmhhcHMgdGhlcmUgd2VyZW4ndCBhcyBtYW55IGZhbWlsaWVzIHdpdGggY2hpbGRyZW4gb24gYm9hcmQhKQo2LiBgRW1iYXJrZWRgOiBTb3V0aGFtcHRvbiBoYXMgYSBoaWdoZXIgZmF0YWxpdHkgcmF0ZSB0aGFuIENoZXJib3VyZyBvciBRdWVlbnN0b3duLiBBIGNyb3NzLXRhYnVsYXRpb24gYmV0d2VlbiBgRW1iYXJrZWRgIGFuZCBgUGNsYXNzYCBzaG93cyB0aGF0IDcyJSBvZiB0aGUgM3JkIGNsYXNzIHBhc3NlbmdlcnMgYW5kIDg5JSBvZiB0aGUgMm5kIGNsYXNzIHBhc3NlbmdlcnMgYm9hcmRlZCBhdCBTb3V0aGFtcHRvbi4gVGhpcyBqaXZlcyB3aXRoIHRoZSBvYnNlcnZhdGlvbiB0aGF0IDJuZCBhbmQgM3JkIGNsYXNzIHBhc3NlbmdlcnMgaGF2ZSBoaWdoZXIgZmF0YWxpdHkgcmF0ZXMuCgpgYGB7ciwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KZ2V0X2xlZ2VuZDwtZnVuY3Rpb24obXlnZ3Bsb3QpewogIHRtcCA8LSBnZ3Bsb3RfZ3RhYmxlKGdncGxvdF9idWlsZChteWdncGxvdCkpCiAgbGVnIDwtIHdoaWNoKHNhcHBseSh0bXAkZ3JvYnMsIGZ1bmN0aW9uKHgpIHgkbmFtZSkgPT0gImd1aWRlLWJveCIpCiAgbGVnZW5kIDwtIHRtcCRncm9ic1tbbGVnXV0KICByZXR1cm4obGVnZW5kKQp9CnAgPC0gbGFwcGx5KFggPSBjKCdQY2xhc3MnLCdTZXgnLCdTaWJTcCcsJ1BhcmNoJywnRW1iYXJrZWQnKSwKICAgICAgICAgICAgRlVOID0gZnVuY3Rpb24oeCkgZ2dwbG90KGRhdGEgPSB0cmFpbi5yYXcpKwogICAgICAgICAgICAgICAgYWVzX3N0cmluZyh4PXgsZmlsbD0nU3Vydml2ZWQnKSsKICAgICAgICAgICAgICAgIGdlb21fYmFyKHBvc2l0aW9uPSJkb2RnZSIpKwogICAgICAgICAgICAgICAgdGhlbWUobGVnZW5kLnBvc2l0aW9uPSJub25lIikpCmxlZ2VuZCA8LSBnZXRfbGVnZW5kKGdncGxvdChkYXRhID0gdHJhaW4ucmF3LGFlcyh4PVBjbGFzcyxmaWxsPVN1cnZpdmVkKSkrZ2VvbV9iYXIoKSkKZ3JpZC5hcnJhbmdlKHBbWzFdXSxwW1syXV0scFtbM11dLHBbWzRdXSxwW1s1XV0sbGVnZW5kLGxheW91dF9tYXRyaXggPQogICAgICAgICAgICAgICAgIGNiaW5kKGMoMSwyLDMpLGMoNCw1LE5BKSxjKDYsNiw2KSksd2lkdGhzPWMoMywzLDEpKQojIHJvdW5kKHByb3AudGFibGUodGFibGUodHJhaW4ucmF3JEVtYmFya2VkLHRyYWluLnJhdyRQY2xhc3MpLG1hcmdpbiA9IDIpLDIpCmBgYAoKIyMjIE11bHRpdmFyaWF0ZSBBbmFseXNlcwoKR3JvdXBlZCBib3hwbG90cyBhcmUgYSBjb21tb24gbWV0aG9kIG9mIGNvbXBhcmluZyBkaXN0cmlidXRpb25zIGdyb3VwZWQgYnkgY2F0ZWdvcmljYWwgdmFyaWFibGVzLiBJIGZpbmQgW2JlYW5wbG90c10oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2JlYW5wbG90L2JlYW5wbG90LnBkZikgdG8gYmUgZXhjZWxsZW50IGNvbXBsZW1lbnRhcnkgcGxvdHMgdG8gYm94cGxvdHMgKGFuZCBpbiBzb21lIGNhc2VzLCBldmVuIGJldHRlcikuIFRoZXkncmUgYSBiaXQgdHJpY2t5IHRvIHJlYWQgYXQgZmlyc3QgLSBzaW5jZSB0aGV5IGFyZSBzbyB1bmRlcnV0aWxpemVkIC0gYnV0IGp1c3QgdGhyb3VnaCBvbmUgcGxvdCwgYSB3ZWFsdGggb2YgaW5mb3JtYXRpb24gY2FuIGJlIGV4dHJhY3RlZC5bXjNdCgpIZXJlIGlzIGEgY29tcGFyaXNvbiBvZiB0aGUgc2FtZSBpbmZvcm1hdGlvbiBiZXR3ZWVuIGEgYm94cGxvdCBhbmQgYSBiZWFucGxvdC4gV2hhdCBjYW4gd2UgaW5mZXIgZnJvbSB0aGUgYmVhbiBwbG90IGJldHRlcj8KCjEuIFRoZSBiZWFucGxvdCBhbGxvd3MgdXMgdG8gdmlzdWFsaXplIHRoZSBkZW5zaXR5IGZ1bmN0aW9uIG9mIHRoZSBwYXJhbWV0ZXIsIGluIHRoaXMgY2FzZTogQWdlLiBGdXJ0aGVybW9yZSwgdGhlIGxlbmd0aCBvZiBlYWNoIGJlYW5saW5lIGlzIGN1bXVsYXRpdmUgdG8gdGhlIG51bWJlciBvZiBkYXRhcG9pbnRzIHRoYXQgZXhpc3QuIFJpZ2h0YXdheSwgd2UgY2FuIHRlbGwgdGhhdCBQY2xhc3M9MyBoYXMgdGhlIG1vc3QgZGF0YSBpbiB0aGUgc2V0LCB3aXRoIHNwYXJzZXIgZGF0YSBhdCBQY2xhc3M9MS4KMi4gVGhlIG1lYW4gdmFsdWVzIGZvciAxc3QgY2xhc3MgaXMgaGlnaGVyIHRoYW4gdGhhdCBmb3IgMm5kIGFuZCAzcmQgY2xhc3MuIFRoZSBkaXN0cmlidXRpb25zIG9mIGRlY2Vhc2VkIGFuZCBzdXJ2aXZlZCBmb3IgMXN0IGNsYXNzIGFyZSBmYWlybHkgc2ltaWxhci4KMy4gRm9yIDJuZCBhbmQgM3JkIGNsYXNzLCB0aGUgc3Vydml2ZWQgZGF0YSBzaG93cyBhIGJpbW9kYWwgZGlzdHJpYnV0aW9uLiBCdW1wcyBhdCB0aGUgMC0xMCBhZ2Ugc2hvdyB0aGF0IGNoaWxkcmVuIHdlcmUgZXZhY3VhdGVkIGZpcnN0LiBUaGlzIGlzIGFsc28gdGhlIHJlYXNvbiB0aGUgbWVhbiB2YWx1ZXMgZm9yIHN1cnZpdmVkIGlzIGxvd2VyLgo0LiBGb3IgMm5kIGFuZCAzcmQgY2xhc3MsIHRoZSBkZWNlYXNlZCBkYXRhIHNob3dzIGEgZmFpcmx5IG5vcm1hbCBkaXN0cmlidXRpb24uCjUuIFRoZSBpbmRpdmlkdWFsIG1lYXN1cmVtZW50cyAocmVwcmVzZW50ZWQgYnkgYmxhY2sgbGluZXMpIHJlcHJlc2VudCBlYWNoIG9ic2VydmF0aW9uIGFuZCBoZWxwIGlkZW50aWZ5IG91dGxpZXJzIG11Y2ggbW9yZSBlYXNpbHkgdGhhbiBhIGJveHBsb3QgZG9lcy4KClteM106IFJlYWQgbW9yZSBhYm91dCBiZWFucGxvdHMgaGVyZTogaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2JlYW5wbG90L3ZpZ25ldHRlcy9iZWFucGxvdC5wZGYKCmBgYHtyLCBmaWcuaGVpZ2h0PTMsIGZpZy53aWR0aD01LCBtZXNzYWdlPUZBTFNFLCB3YXJuaW5nPUZBTFNFfQpnZ3Bsb3QodHJhaW4ucmF3LGFlcyh5PUFnZSx4PVBjbGFzcykpK2dlb21fYm94cGxvdChhZXMoZmlsbD1TdXJ2aXZlZCkpK3RoZW1lX2J3KCkKYmVhbnBsb3QoQWdlflN1cnZpdmVkKlBjbGFzcyxzaWRlPSdiJyx0cmFpbi5yYXcsY29sPWxpc3QoJ3llbGxvdycsJ29yYW5nZScpLAogICAgICAgICBib3JkZXIgPSBjKCd5ZWxsb3cyJywnZGFya29yYW5nZScpLGxsID0gMC4wNSxib3h3ZXggPSAuNSwKICAgICAgICAgbWFpbj0nUGFzc2VuZ2VyIHN1cnZpdmFsIGJ5IHBjbGFzcyBhbmQgQWdlJyx4bGFiPSdQYXNzZW5nZXIgQ2xhc3MnLHlsYWI9J0FnZScpCmxlZ2VuZCgndG9wcmlnaHQnLCBmaWxsID0gYygneWVsbG93Jywnb3JhbmdlJyksIGxlZ2VuZCA9IGMoIkRlYWQiLCAiU3Vydml2ZWQiKSxidHkgPSAnbicsY2V4ID0gLjgpCmBgYAoKQSBsb29rIGludG8gdGhlIGBTaWJTcGAgYW5kIGBQYXJjaGAgdmFyaWFibGVzIHNob3dzIHNvbWV0aGluZyBpbnRlcmVzdGluZy4gVGhlcmUgYXJlIHRocmVlIHJlZ2lvbnMgb25lIGNhbiBpZGVudGlmeToKCiogVGhlIHByb2JhYmlsaXR5IG9mIHN1cnZpdmFsIGlzIG1pbmltYWwgZm9yIG51bWJlciBvZiBwYXJlbnRzL2NoaWxkcmVuIGFib2FyZCA+IDMuCiogVGhlIHByb2JhYmlsaXR5IG9mIHN1cnZpdmFsIGlzIG1pbmltYWwgZm9yIG51bWJlciBvZiBzaWJsaW5ncy9zcG91c2VzIGFib2FyZCA+IDMuCiogRm9yIGBTaWJTcGA8PTMgYW5kIGBQYXJjaGA8PTMsIHRoZXJlIGFyZSBiZXR0ZXIgY2hhbmNlcyBmb3Igc3Vydml2YWwuCgpUaGUgZ3JvdXBpbmcgYnkgYFBjbGFzc2AgcmV2ZWFscyB0aGF0IGFsbCB0aGUgbGFyZ2UgZmFtaWxpZXMgd2VyZSAzcmQgY2xhc3MgdHJhdmVsZXJzLiBXb3JzZSBhY2Nlc3MgdG8gaGVscC4uLiBsb3dlc3QgY2hhbmNlIGZvciBzdXJ2aXZhbC4KClRoZXNlIGNvdWxkIGJlIHNpbXBsZSBydWxlcyBlaXRoZXIgaGFyZCBjb2RlZCBkdXJpbmcgbW9kZWwgYnVpbGRpbmc6IHNvbWV0aGluZyBhbG9uZyB0aGUgbGluZXMgb2Y6ICpJRiAoU2liU3A+MyBPUiBQYXJjaCA+MykgVEhFTiBwcmVkaWN0aW9uID0gMCosIG9yIHNvbWUgZGVyaXZlZCB2YXJpYWJsZXMgY2FuIGJlIGNyZWF0ZWQuCgpgYGB7cn0KZ2dwbG90KHRyYWluLnJhdyxhZXMoeT1TaWJTcCx4PVBhcmNoKSkrCiAgICBnZW9tX2ppdHRlcihhZXMoY29sb3I9U3Vydml2ZWQsc2hhcGU9UGNsYXNzKSkrCiAgICB0aGVtZV9idygpKwogICAgc2NhbGVfc2hhcGUoc29saWQ9RikrCiAgICBnZW9tX3ZsaW5lKHhpbnRlcmNlcHQgPSAzLGNvbG9yPSdkYXJrYmx1ZScsbHR5PTMpKwogICAgZ2VvbV9obGluZSh5aW50ZXJjZXB0ID0gMyxjb2xvcj0nZGFya2JsdWUnLGx0eT0zKQpgYGAKCioqKgoKIyBEYXRhIFByZXBhcmF0aW9uCgojIyBNaXNzaW5nIFZhbHVlcyBJbXB1dGF0aW9uClN0YXJ0aW5nIHdpdGggdGhlIGVhc2llciBvbmUgZmlyc3Q6CgoqKkVtYmFya2VkKio6IFRoZSBsYXJnZXN0IHBvcnRpb24gb2YgdGhlIHBhc3NlbmdlcnMgZW1iYXJlZCBhdCBTb3V0aGhhbXB0b24uIEknbSByZXBsYWNpbmcgdGhlIE5BcyB3aXRoIHRoZSBzYW1lLiBGaXJzdCwgSSBjcmVhdGUgYSBuZXcgaW1wdXRlZCB0cmFpbmluZyBkYXRhc2V0LgoKYGBge3J9CnN1bW1hcnkodHJhaW4ucmF3JEVtYmFya2VkKQp0cmFpbi5pbXAgPC0gdHJhaW4ucmF3CnRyYWluLmltcCRFbWJhcmtlZFtpcy5uYSh0cmFpbi5pbXAkRW1iYXJrZWQpXSA8LSAnUycKYGBgCgoqKk5hbWVzLCBUaXRsZXMgJiBBZ2UqKjoKClRoZSBuYW1lcyBoYXZlIHRpdGxlcyBlbWJlZGRlZCBpbiB0aGUgc3RyaW5ncy4gSSBjYW4gZXh0cmFjdCB0aGVzZSB1c2luZyByZWdleC4gTWFzdGVyLCBNaXNzLCBNciBhbmQgTXJzIGFyZSB0aGUgbW9zdCBwb3B1bGFyIC0gbm8gc3VycHJpc2UgdGhlcmUsIHdpdGggbG90cyBvZiBvdGhlciB0aXRsZXMuICBIZXJlJ3MgdGhlIGRpc3RyaWJ1dGlvbiBvZiB0aGUgdGl0bGVzIGJ5IGFnZS4gVGhlc2UgY2FuIGJlIHVzZWQgdG8gaW1wdXRlIHRoZSBtaXNzaW5nIGFnZSB2YWx1ZXMuCgpgYGB7ciwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KdHJhaW4ucmF3JHRpdGxlIDwtIHN0cl9leHRyYWN0KHBhdHRlcm4gPSAnW2EtekEtWl0rKD89XFwuKScsc3RyaW5nID0gdHJhaW4ucmF3JE5hbWUpCnRyYWluLnJhdyR0aXRsZSA8LSBhcy5mYWN0b3IodHJhaW4ucmF3JHRpdGxlKQoKdHJhaW4ucmF3ICU+JQogICAgbmEub21pdCgpICU+JQogICAgZ3JvdXBfYnkodGl0bGUpICU+JQogICAgZHBseXI6OnN1bW1hcmlzZShDb3VudD1uKCksIE1lZGlhbl9BZ2U9cm91bmQobWVkaWFuKEFnZSksMCkpICU+JQogICAgYXJyYW5nZSgtTWVkaWFuX0FnZSkKYGBgCgpgYGB7ciwgbWVzc2FnZT1GQUxTRSwgd2FybmluZz1GQUxTRX0KZ2dwbG90KHRyYWluLnJhdyxhZXMoeD10aXRsZSx5PUFnZSkpKwogICAgc3RhdF9zdW1tYXJ5KGFlcyh5ID0gQWdlLGdyb3VwPTEpLCBmdW4ueT1tZWRpYW4sIGNvbG91cj0icmVkIiwgZ2VvbT0icG9pbnQiLGdyb3VwPTEpKwogICAgZ2VvbV9qaXR0ZXIoc2hhcGU9MjEsYWxwaGE9LjYsY29sPSdibHVlJykrCiAgICB0aGVtZV9idygpKwogICAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSxsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICAgIGxhYnMoY2FwdGlvbj0nUmVkIHBvaW50cyBhcmUgbWVkaWFuIHZhbHVlcycpCmBgYAoKR3JvdXBpbmcgc2ltaWxhciB0aXRsZXMgdG9nZXRoZXIsIEkndmUga2VwdCBhIGZldyB0aXRsZXMgLSBPZmZpY2VyLCBSb3lhbHR5LCBNciwgTXJzIGFuZCBNaXNzLgoKYGBge3J9CnRyYWluLmltcCA8LSB0cmFpbi5yYXcKdHJhaW4uaW1wJHRpdGxlIDwtIGFzLmNoYXJhY3Rlcih0cmFpbi5pbXAkdGl0bGUpCnRyYWluLmltcCR0aXRsZVt0cmFpbi5pbXAkdGl0bGUgJWluJSBjKCdDYXB0JywnQ29sJywnTWFqb3InKV0gPC0gJ09mZmljZXInCnRyYWluLmltcCR0aXRsZVt0cmFpbi5pbXAkdGl0bGUgJWluJSBjKCdEb24nLCdEcicsJ1JldicsJ1NpcicsJ0pvbmtoZWVyJywnQ291bnRlc3MnLCdMYWR5JywnRG9uYScpXSA8LSAnUm95YWx0eScKdHJhaW4uaW1wJHRpdGxlW3RyYWluLmltcCR0aXRsZSAlaW4lIGMoJ01ycycsJ01tZScpXSA8LSAnTXJzJwp0cmFpbi5pbXAkdGl0bGVbdHJhaW4uaW1wJHRpdGxlICVpbiUgYygnTXMnLCdNbGxlJyldIDwtICdNaXNzJwp0cmFpbi5pbXAkdGl0bGUgPC0gYXMuZmFjdG9yKHRyYWluLmltcCR0aXRsZSkKCnRyYWluLmltcCAlPiUKICAgIGdyb3VwX2J5KHRpdGxlKSAlPiUKICAgIHN1bW1hcmlzZShNZWRpYW5fQWdlPW1lZGlhbihBZ2UsbmEucm0gPSBUKSkKYGBgCgpgYGB7cn0KZ2dwbG90KHRyYWluLmltcCxhZXMoeD10aXRsZSx5PUFnZSkpKwogICAgZ2VvbV9qaXR0ZXIoc2hhcGU9MjEsYWxwaGE9LjYsY29sPSdibHVlJykrCiAgICBzdGF0X3N1bW1hcnkoYWVzKHkgPSBBZ2UsZ3JvdXA9MSksIGZ1bi55PW1lZGlhbiwgY29sb3VyPSJyZWQiLCBnZW9tPSJwb2ludCIsZ3JvdXA9MSkrCiAgICB0aGVtZV9idygpKwogICAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSxsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICAgIGxhYnMoY2FwdGlvbj0nUmVkIHBvaW50cyBhcmUgbWVkaWFuIHZhbHVlcycpCmBgYAoKTm93IGZvciB0aGUgbWlzc2luZyBBZ2UgdmFsdWVzLiBJJ20gdHJ5aW5nIG91dCB0d28gc3RyYXRlZ2llcyB0byBpbXB1dGUgYWdlLCBqdXN0IGZvciBraWNrcy4gRmlyc3QsIGEgcmVncmVzc2lvbiB0cmVlIHVzaW5nIHRoZSBgcnBhcnRgIG1ldGhvZC4gNS1yZXBlYXQgMTAtZm9sZCBjcm9zcyB2YWxpZGF0aW9uIGFjcm9zcyBhIHR1bmluZyBncmlkIG9mIDIwIHZhbHVlcyBvZiBgbWF4ZGVwdGhgLiBSTVNFIHN0YWJsaXplcyBhdCBhIGRlcHRoIG9mIDE0LCB3aXRoIGEgdmFsdWUgb2YgMTIuMi4KCmBgYHtyfQphZ2UucHJlZGljdG9ycyA8LSB0cmFpbi5pbXAgJT4lCiAgICBkcGx5cjo6c2VsZWN0KC1TdXJ2aXZlZCwtQ2FiaW4sLVRpY2tldCwtTmFtZSkgJT4lCiAgICBmaWx0ZXIoY29tcGxldGUuY2FzZXMoLikpCnNldC5zZWVkKDEyMzQpCmN0cmwgPC0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJib290IiwKICAgICAgICAgICAgICAgICAgICAgcmVwZWF0cyA9IDUsCiAgICAgICAgICAgICAgICAgICAgIG51bWJlciA9IDIwMAogICAgICAgICAgICAgICAgICAgICApCnJwYXJ0R3JpZCA8LSBkYXRhLmZyYW1lKG1heGRlcHRoID0gc2VxKDQsMjAsMikpCnJwYXJ0Rml0IDwtIHRyYWluKEFnZX4uLAogICAgICAgICAgICAgICAgICBkYXRhPWFnZS5wcmVkaWN0b3JzLAogICAgICAgICAgICAgICAgICBtZXRob2Q9J3JwYXJ0MicsCiAgICAgICAgICAgICAgICAgIHRyQ29udHJvbCA9IGN0cmwsCiAgICAgICAgICAgICAgICAgIHR1bmVHcmlkID0gcnBhcnRHcmlkCiAgICAgICAgICAgICAgICAgICkKcnBhcnRGaXQKcGxvdChycGFydEZpdCkKcGxvdChycGFydEZpdCRmaW5hbE1vZGVsLG1hcmdpbj0wLjAyKQp0ZXh0KHJwYXJ0Rml0JGZpbmFsTW9kZWwsY2V4PTAuOCkKYGBgCgpBbm90aGVyIHdheSBpcyB0byBydW4gYSByYW5kb21mb3Jlc3Qgd2l0aCBhIHNlYXJjaCBvdmVyIHZhbHVlcyBvZiBgbXRyeWAgdXNpbmcgNS1yZXBlYXQgMTAtZm9sZCBjcm9zcyB2YWxpZGF0aW9uLiBBcyB3ZSBjYW4gc2VlIG10cnk9NCBpcyB0aGUgb3B0aW1hbCB2YWx1ZSB3aGljaCByZXN1bHRzIGluIHRoZSBsb3dlc3QgUk1TRSBvZiAxMS40OyBtdWNoIGJldHRlciB0aGFuIHRoZSBycGFydCBtb2RlbC4KCmBgYHtyfQpzZXQuc2VlZCgxMjM0KQpyZkdyaWQgPC0gZGF0YS5mcmFtZShtdHJ5PXNlcSgxLDYsMSkpCmN0cmwgPC0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJyZXBlYXRlZGN2IiwKICAgICAgICAgICAgICAgICAgICAgcmVwZWF0cyA9IDUKICAgICAgICAgICAgICAgICAgICAgKQpyZkZpdCA8LSB0cmFpbihBZ2V+LiwKICAgICAgICAgICAgICAgICAgZGF0YT1hZ2UucHJlZGljdG9ycywKICAgICAgICAgICAgICAgICAgbWV0aG9kPSdyZicsCiAgICAgICAgICAgICAgICAgIHRyQ29udHJvbCA9IGN0cmwsCiAgICAgICAgICAgICAgICAgIHR1bmVHcmlkID0gcmZHcmlkKQpyZkZpdApwbG90KHJmRml0KQpgYGAKCkknbSBnb2luZyB0byB1c2UgdGhlIHJhbmRvbUZvcmVzdCBtb2RlbC4gVXNpbmcgdGhlIGBwcmVkaWN0LnRyYWluKClgIHRvIHByZWRpY3QgdmFsdWVzIG9mIGFnZSBhbmQgcGx1ZyB0aGVtIGJhY2sgaW50byB0aGUgaW1wdXRlZCBkYXRhLiBZb3UgY2FuIHNlZSB0aGUgYmx1ZSBwb2ludHMgd2hpY2ggYXJlIHRoZSBpbXB1dGVkIHZhbHVlcyBvZiBgQWdlYC4gV2hhdCBJIG5vdGljZWQgaXMgdGhhdCBmb3IgYWxsIHRoZSB0aXRsZXMsIHRoZSBpbXB1dGVkIEFnZSB2YWx1ZSBzZWVtcyB0byBiZSBkaXN0cmlidXRlZCBmYWlybHkgd2VsbCwgZXhjZXB0IE1hc3Rlci4gRm9yIE1hc3RlciwgdGhlIHRocmVlIGltcHV0ZWQgYXJlIGRlZmluaXRlbHkgb3V0bGllcnMuIEknbSBnb2luZyB0byBmb3JjZSB0aGVzZSB0byB0aGUgbWVkaWFuIEFnZS4KCmBgYHtyfQptaXNzaW5nLmFnZSA8LSB0cmFpbi5pbXAgJT4lIGZpbHRlcihpcy5uYShBZ2UpKQphZ2UucHJlZGljdGVkIDwtIHByZWRpY3QocmZGaXQsIG5ld2RhdGEgPSBtaXNzaW5nLmFnZSkKdHJhaW4uaW1wICU+JQogICAgbXV0YXRlKEFnZU1pc3NpbmcgPSBpcy5uYShBZ2UpLAogICAgICAgICAgIEFnZSA9IGlmZWxzZShBZ2VNaXNzaW5nLGFnZS5wcmVkaWN0ZWQsQWdlKSkgJT4lCiAgICBnZ3Bsb3QoYWVzKHg9dGl0bGUseT1BZ2UpKSsKICAgIHN0YXRfc3VtbWFyeShhZXMoeSA9IEFnZSxncm91cD0xKSwgZnVuLnk9bWVkaWFuLCBjb2xvdXI9InJlZCIsIGdlb209InBvaW50Iixncm91cD0xKSsKICAgIGdlb21faml0dGVyKGFlcyh5PUFnZSxjb2w9QWdlTWlzc2luZyksc2hhcGU9MikrCiAgICB0aGVtZV9idygpKwogICAgdGhlbWUoYXhpcy50ZXh0LnggPSBlbGVtZW50X3RleHQoYW5nbGUgPSA0NSwgaGp1c3QgPSAxKSxsZWdlbmQucG9zaXRpb249Im5vbmUiKSsKICAgIGxhYnMoY2FwdGlvbj0nUmVkIHBvaW50cyBhcmUgbWVkaWFuIHZhbHVlcycpCnRyYWluLmltcCRBZ2VbaXMubmEodHJhaW4uaW1wJEFnZSldIDwtIGFnZS5wcmVkaWN0ZWQKdHJhaW4uaW1wJEFnZVt0cmFpbi5pbXAkdGl0bGU9PSdNYXN0ZXInICYgdHJhaW4uaW1wJEFnZSA+IDIwXSA8LSBtZWRpYW4odHJhaW4uaW1wJEFnZVt0cmFpbi5pbXAkdGl0bGU9PSdNYXN0ZXInXSxuYS5ybSA9IFQpCmBgYAoKIyMgRGVyaXZlZCBWYXJpYWJsZXMKCgoqKkNoaWxkPzoqKiBUcnlpbmcgb3V0IHR3byBlbmdpbmVlcmVkIHZhcmlhYmxlcyBoZXJlIC0gaXMgdGhlIHBhc3NlbmdlciBhIGNoaWxkIG9yIG5vdD8gVXNpbmcgQWdlPTE4IGFzIGEgdGhyZXNob2xkLiBBbmQgaXMgcy9oZSBjbG9zZSBlbm91Z2ggdG8gYmUgY29uc2lkZXJlZCBhIGFkdWx0IGJ5IGNoYW5jZT8gVGhvc2UgYmV0d2VlbiAxNiBhbmQgMTggY291bGQgYmUgbWlzdGFrZW4gZm9yIG5vdCBiZWluZyBjaGlsZHJlbi4gKE15IHdheSBvZiBpbmNvcnBvcmF0aW5nIGEgZnVkZ2UgZmFjdG9yIGluIHRoZSBkZWNpc2lvbiBwcm9jZXNzIG9mIGxhZGllcyAmIGNoaWxkcmVuIGZpcnN0LikKCmBgYHtyfQp0cmFpbi5pbXAkY2hpbGQgPC0gMAp0cmFpbi5pbXAkY2hpbGRbdHJhaW4uaW1wJEFnZTwxOF0gPC0gMQp0cmFpbi5pbXAkYWxtb3N0YWR1bHQgPC0gYXMubnVtZXJpYyhiZXR3ZWVuKHRyYWluLmltcCRBZ2UsMTYsMTgpKQpgYGAKCioqUmVhbGx5IHlvdW5nLCBvciByZWFsbHkgb2xkPzoqKiBSZWFsbHkgeW91bmcgb25lcyBhbmQgb2xkZXIgZm9sa3Mgd291bGQgZ2V0IHByaW9yaXR5IHBlcmhhcHMuIENyZWF0aW5nIHR3byBjYXRlZ29yaWNhbCBiaW5hcnkgdmFyaWFibGVzIGZvciB0aGVzZSBjb25kaXRpb25zLgpgYGB7cn0KdHJhaW4uaW1wJFlvdW5nIDwtIGlmZWxzZSh0cmFpbi5pbXAkQWdlPDEwLDEsMCkKdHJhaW4uaW1wJFNlbmlvcnMgPC0gaWZlbHNlKHRyYWluLmltcCRBZ2U+NjAsMSwwKQpgYGAKCgoqKkZhbWlseSByZWxhdGVkOioqIExldCdzIGFsc28gY3JlYXRlIHNvbWUgdmFyaWFibGVzIHRoYXQgdGFsayBhYm91dCBmYW1pbHkgc2l6ZXMuIFdoYXQncyB0aGUgdG90YWwgZmFtaWx5IHNpemUgLS0gY29udGlub3VzIHZhcmlhYmxlIGBUb3RhbEZhbWAuIElzIHRoZSBwZXJzb24gc2luZ2xlLCBwYXJ0IG9mIGEgY291cGxlIG9yIGEgZmFtaWx5PyBUaHJlZSBjYXRlZ29yaWNhbCB2YXJpYWJsZXMgZm9yIHRoZXNlLgoKYGBge3J9CnRyYWluLmltcCRUb3RhbEZhbSA8LSB0cmFpbi5pbXAkU2liU3AgKyB0cmFpbi5pbXAkUGFyY2ggKyAxCiMgdHJhaW4uaW1wJExhc3ROYW1lIDwtIHRyYWluLmltcCROYW1lICU+JSBzdHJfZXh0cmFjdChwYXR0ZXJuID0gJ1thLXpBLVpdKyg/PSwpJykKIyB0cmFpbi5pbXAkRmFtU2l6ZSA8LSBwYXN0ZTAodHJhaW4uaW1wJFRvdGFsRmFtLHRyYWluLmltcCRMYXN0TmFtZSkKIyB0cmFpbi5pbXAkTGFzdE5hbWUgPC0gTlVMTAp0cmFpbi5pbXAkTmFtZSA8LSBOVUxMCnRyYWluLmltcCRMYXJnZVBhckNoIDwtIGFzLm51bWVyaWModHJhaW4uaW1wJFBhcmNoPj0zKQp0cmFpbi5pbXAkTGFyZ2VTaWJTcCA8LSBhcy5udW1lcmljKHRyYWluLmltcCRTaWJTcD49MykKdHJhaW4uaW1wJFNpbmdsZSA8LSBpZmVsc2UodHJhaW4uaW1wJFRvdGFsRmFtPT0xLDEsMCkKdHJhaW4uaW1wJENvdXBsZSA8LSBpZmVsc2UodHJhaW4uaW1wJFRvdGFsRmFtPT0yLDEsMCkKdHJhaW4uaW1wJEZhbWlseSA8LSBpZmVsc2UodHJhaW4uaW1wJFRvdGFsRmFtPjIsMSwwKQpgYGAKCioqQ2FiaW4gcmVsYXRlZDoqKiBFeHRyYWN0aW5nIHRoZSBjYWJpbiBhbHBoYWJldCBhbmQgbnVtYmVyIGZyb20gdGhlIGNhYmluIHZhcmlhYmxlLiBTaW5jZSB0aGUgY2FiaW4gbnVtYmVycyBjb3VsZCBiZSBvcmRlcmVkIGZyb20gbGVmdCB0byByaWdodCBvciB0b3AgdG8gYm90dG9tIG9uIHRoZSBib2F0LCBwZXJoYXBzIG9ubHkgdGhlIDFzdCBkaWdpdCBpcyBzaWduaWZpY2FudC4gQWxzbywgc29tZSBmb2xrcyBoYXZlIG1vcmUgdGhhbiAxIGNhYmluLiBXb25kZXIgaWYgdGhhdCdzIGltcG9ydGFudC4gU2luY2UgbG90cyBvZiB1bmtub3ducyBpbiB0aGUgYENhYmluYCB2YXJpYWJsZSwgYWxsIE5BIHZhbHVlcyBhcmUgcmVwbGFjZWQgYnkgJ1UnLiBSZWZlcmluZyB0byB0aGUgZGVjayBkaWFncmFtLCB0aGUgdG9wbW9zdCBkZWNrcyBhcmUgQSBhbmQgQiwgd2hpY2ggYXJlIGNsb3Nlc3QgdG8gdGhlIGxpZmVib2F0cy4gUGVyaGFwcyB0aGF0J3MgaW1wb3J0YW50IHRvby4gSGVyZSwgSSBjcmVhdGUgYSBidW5jaCBvZiBjYXRlZ29yaWNhbCB2YXJpYWJsZXMgYmFzZWQgb2ZmIHRoZSBvcmlnaW5hbCBgQ2FiaW5gLCBhbmQgdGhlbiByZW1vdmUgaXQgZnJvbSB0aGUgZGF0YXNldC4KCmBgYHtyfQp0cmFpbi5pbXAkQ2FiaW5NaXNzaW5nIDwtIGFzLm51bWVyaWMoaXMubmEodHJhaW4ucmF3JENhYmluKSkKCnRyYWluLmltcCRDYWJpbkNvZGUgPC0gbWFwX2Nocih0cmFpbi5yYXckQ2FiaW4sfnN0cl9zcGxpdChzdHJpbmcgPSAueCxwYXR0ZXJuID0gJycpW1sxXV1bMV0pCnRyYWluLmltcCRDYWJpbkNvZGVbaXMubmEodHJhaW4uaW1wJENhYmluQ29kZSldIDwtICdVJwoKdHJhaW4uaW1wJENhYmluTnVtIDwtIGFzLm51bWVyaWMobWFwX2Nocih0cmFpbi5yYXckQ2FiaW4sfnN0cl9zcGxpdChzdHJpbmcgPSAueCxwYXR0ZXJuID0gJ1thLXpBLVpdJylbWzFdXVsyXSkpCnRyYWluLmltcCRDYWJpbk51bSA8LSBtYXBfaW50KHRyYWluLmltcCRDYWJpbk51bSwgfmFzLmludGVnZXIoc3RyX3NwbGl0KC54LHBhdHRlcm4gPSAnJyxzaW1wbGlmeSA9IFQpWzFdWzFdKSkKdHJhaW4uaW1wJENhYmluTnVtW2lzLm5hKHRyYWluLmltcCRDYWJpbk51bSldIDwtIDAKCnRyYWluLmltcCRUb3BEZWNrIDwtIGlmZWxzZSh0cmFpbi5pbXAkQ2FiaW5Db2RlICVpbiUgYygnQScsJ0InKSwxLDApCnRyYWluLmltcCRNaWREZWNrIDwtIGlmZWxzZSh0cmFpbi5pbXAkQ2FiaW5Db2RlICVpbiUgYygnQycsJ0QnKSwxLDApCnRyYWluLmltcCRMb3dlckRlY2sgPC0gaWZlbHNlKHRyYWluLmltcCRUb3BEZWNrPT0wICYgdHJhaW4uaW1wJE1pZERlY2sgPT0wICwxLDApCgp0cmFpbi5pbXAkTnVtYmVyb2ZDYWJpbnMgPC0gbWFwX2ludCh0cmFpbi5yYXckQ2FiaW4sfnN0cl9zcGxpdChzdHJpbmcgPSAueCxwYXR0ZXJuID0gJyAnKVtbMV1dICU+JSBsZW5ndGgpCnRyYWluLmltcCRDYWJpbiA8LSBOVUxMCmBgYAoKKipUaWNrZXQ6KiogTGFzdGx5LCB0aGUgYHRpY2tldGAgdmFyaWFibGUuIEknbSBub3Qgc3VyZSB3aGF0IHRvIG1ha2Ugb2YgaXQsIHNvIEknbSBrZWVwaW5nIGl0IGZvciBub3csIGFmdGVyIGNsZWFuaW5nIGl0IHVwIGEgYml0LiBBIG1ham9yaXR5ICg4MCUpIG9mIHRoZSByb3dzIGhhdmUgdW5pcXVlIChvbmUpIHRpY2tldC4gMTQlIHJvd3MgaGF2ZSBhIGR1cGxpY2F0ZSB0aWNrZXQsIHBlcmhhcHMgaW5kaWNhdGluZyBhIGZhbWlseS4gQSBzbWFsbCBudW1iZXIgb2Ygcm93cyBoYXZlIDMrIGR1cGxpY2F0ZXMgb2YgdGhlIHRpY2tldHMuCgpgYGB7cn0KdHJhaW4uaW1wJFRpY2tldCAlPiUgdGFibGUoKSAlPiUgYXMubnVtZXJpYygpICU+JSB0YWJsZSgpCmBgYAoKVGhlcmUgc2VlbXMgdG8gYmUgYSBiaXQgb2YgYSBwYXR0ZXJuIGhlcmUuIFRpY2tldHMgc3RhcnRpbmcgd2l0aCAxIGFyZSBtb3N0bHkgMXN0IGNsYXNzLCB0aG9zZSBzdGFydGluZyB3aXRoIDIgYXJlIDJuZCBjbGFzcywgYW5kIDMgLSAzcmQgY2xhc3MuIEJ1dCwgSSBmZWVsIGl0J3MgYSB2ZXJ5IGxvb3NlIGFzc29jaWF0aW9uLgpgYGB7cn0KdHJhaW4uaW1wICU+JSBncm91cF9ieShQY2xhc3MpICU+JSBkcGx5cjo6c2VsZWN0KFRpY2tldCxQY2xhc3MpICU+JSBzYW1wbGVfbig1KQpgYGAKCldoYXQgSSdtIGdvaW5nIHRvIGRvIGlzIGNsZWFuIHVwIHRoZSBjb2x1bW5zIChyZW1vdmUgc3BlY2lhbCBjaGFyYWN0ZXJzLCBzcGFjZXMgZXRjKSwgdGhlbiBzcGxpdCB0aGUgYFRpY2tldGAgY29sdW1uIGludG8gZm91cjogYFRpY2tldENoYXJgLCBgVGlja2V0TnVtYCxgVGlja2V0TnVtTGVuZ3RoYCwgYFRpY2tldE51bVN0YXJ0YC4gIChVcG9uIHJ1bm5pbmcgdGhlIHNjcmlwdCBhIGZldyB0aW1lcywgSSd2ZSBkZWNpZGVkIHRvIGdldCByaWQgb2YgYFRpY2tldE51bWAsIGJ1dCBJJ20gY29tbWVudGluZyB0aGUgY29kZSBmb3IgZnV0dXJlIHJlZikuIFRoZSBgVGlja2V0Q2hhcmAgdmFyaWFibGUgYXMgdGhpcyBkaXN0cmlidXRpb246CgpgYGB7cn0KdHJhaW4uaW1wICU8PiUKICAgIG11dGF0ZSgKICAgICAgICBUaWNrZXQgPSBzdHJfdG9fdXBwZXIoVGlja2V0KSAlPiUKICAgICAgICAgICAgc3RyX3JlcGxhY2VfYWxsKHBhdHRlcm4gPSByZWdleChwYXR0ZXJuID0gJ1suXFwvXScpLHJlcGxhY2VtZW50ID0gJycpLAogICAgICAgIFRpY2tldE51bSA9IHN0cl9leHRyYWN0KFRpY2tldCxwYXR0ZXJuID0gcmVnZXgoJyhbMC05XSl7Myx9JykpLAogICAgICAgIFRpY2tldE51bVN0YXJ0ID0gbWFwX2ludChUaWNrZXROdW0sfmFzLmludGVnZXIoc3RyX3NwbGl0KC54LHBhdHRlcm4gPSAnJyxzaW1wbGlmeSA9IFQpWzFdKSksCiAgICAgICAgVGlja2V0TnVtTGVuID0gbWFwX2ludChUaWNrZXROdW0sfmRpbShzdHJfc3BsaXQoLngscGF0dGVybiA9ICcnLHNpbXBsaWZ5ID0gVCkpWzJdKSwKICAgICAgICBUaWNrZXRDaGFyID0gc3RyX2V4dHJhY3QoVGlja2V0LHBhdHRlcm4gPSByZWdleCgnXlthLXpBLVovXFwuXSsnKSkgCiAgICAgICAgKSAlPiUKICAgICBtdXRhdGUoCiAgICAgICAgIFRpY2tldENoYXIgPSBtYXBfY2hyKC54PVRpY2tldENoYXIsCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIC5mPX5zdHJfc3BsaXQoc3RyaW5nPS54LCBwYXR0ZXJuID0gJycsc2ltcGxpZnkgPSBUKVsxXSkKICAgICAgICAgKSAlPiUgICAgIAogICAgbXV0YXRlKAogICAgICAgIFRpY2tldENoYXIgPSBpZmVsc2UoaXMubmEoVGlja2V0Q2hhciksJ1UnLFRpY2tldENoYXIpLAogICAgICAgIFRpY2tldE51bVN0YXJ0ID0gaWZlbHNlKGlzLm5hKFRpY2tldE51bVN0YXJ0KSwwLFRpY2tldE51bVN0YXJ0KSwKICAgICAgICBUaWNrZXROdW1MZW4gPSBpZmVsc2UoaXMubmEoVGlja2V0TnVtTGVuKSwwLFRpY2tldE51bUxlbiksCiAgICApCnRyYWluLmltcCRUaWNrZXQgPC0gTlVMTAp0cmFpbi5pbXAkVGlja2V0TnVtIDwtIE5VTEwKdGFibGUodHJhaW4uaW1wJFRpY2tldENoYXIpCnRhYmxlKHRyYWluLmltcCRUaWNrZXROdW1MZW4pCnRhYmxlKHRyYWluLmltcCRUaWNrZXROdW1TdGFydCkKYGBgCgojIyBXaW56b3JpbmcgVmFyaWFibGVzCgpUaGUgYGZhcmVgIHZhcmlhYmxlIGhhcyBvbmUgbWFzc2l2ZSBvdXRsaWVyLiBXaW56b3Jpc2luZyB0aGlzIHZhcmlhYmxlIHVzaW5nIHRoZSA5NXRoIHBlcmNlbnRpbGUgdmFsdWUgYXMgdGhlIGN1dG9mZi4KCmBgYHtyfQpnZ3Bsb3QodHJhaW4uaW1wLGFlcyh4PUZhcmUsZmlsbD1QY2xhc3MpKStnZW9tX2hpc3RvZ3JhbSgpK2ZhY2V0X2dyaWQoUGNsYXNzfi4pCnF1YW50aWxlKHRyYWluLmltcCRGYXJlW3RyYWluLmltcCRQY2xhc3M9PSdQMSddLHByb2JzID0gYyguMSwuMjUsLjUsLjc1LC45NSkpCnRyYWluLmltcCRGYXJlW3RyYWluLmltcCRGYXJlPjIzMl0gPC0gMjMyCmBgYAoKIyMgRmluYWwgRGF0YSBSZXZpZXcKClRoZSBkYXRhc2V0IGlzIG5vdyBwcmVwYXJlZCBmb3IgbW9kZWxpbmcuIEhlcmUncyBhIHF1aWNrIHJldmlldyBvZiB0aGUgZGF0YSBzbyBmYXIuIDI5IHZhcmlhYmxlcyBpbiB0b3RhbC4KCmBgYHtyfQp0cmFpbi5pbXAgJT4lIGdsaW1wc2UoKQpgYGAKCgoqKioKCiMgTW9kZWxpbmcKCkknbSBleHBlcmltZW50aW5nIHdpdGggYSBmZXcgbW9kZWxpbmcgdGVjaG5pcXVlcywgbWFpbmx5IFt4Z2Jvb3N0XShodHRwOi8veGdib29zdC5yZWFkdGhlZG9jcy5pby9lbi9sYXRlc3QvKSwgW2dibV0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2dibS9pbmRleC5odG1sKSwgYW5kIHBlbmFsaXplZCBtb2RlbHMgdXNpbmcgW2dsbW5ldF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL2dsbW5ldC9pbmRleC5odG1sKS4gSSd2ZSBpbXBsZW1lbnRlZCBhbGwgdGhlc2UgbW9kZWxzIHVzaW5nIFtjYXJldF0oaHR0cDovL3RvcGVwby5naXRodWIuaW8vY2FyZXQvaW5kZXguaHRtbCkgd2hpY2ggSSBmaW5kIGFuIGFic29sdXRlbHkgaW5kaXNwZW5zaWJsZSB0b29sa2l0IHRvIHByZXAsIGJ1aWxkLCB0dW5lIGFuZCBleHBsb3JlIG51bWVyb3VzIG1vZGVscyB1c2luZyB2ZXJ5IGZldyBsaW5lcyBvZiBjb2RlLgoKRm9yIGFsbCBtb2RlbHMsIEknbSB1c2luZyBhIDUtcmVwZWF0IDEwLWZvbGQgY3Jvc3MgdmFsaWRhdGlvbiB0ZWNobmlxdWUgb24gdGhlIHRyYWluaW5nIGRhdGFzZXQuIFRodXMsIEkgaGF2ZSBub3Qgc3BsaXQgdGhlIHRyYWluaW5nIGRhdGFzZXQgZnVydGhlciBpbnRvIHRlc3QtdHJhaW4gc2V0cywgZ2l2ZW4gdGhlIHNtYWxsIG51bWJlciBvZiBvYnNlcnZhdGlvbnMgaW4gdGhlIGRhdGFzZXQuIAoKRnVydGhlcm1vcmUsIGdpdmVuIHRoZSA4MDoyMCBjbGFzcy1pbWJhbGFuY2UsIEknbSBhbHNvIHRyeWluZyBvdXQgW3Ntb3RlXShodHRwczovL3d3dy5qYWlyLm9yZy9tZWRpYS85NTMvbGl2ZS05NTMtMjAzNy1qYWlyLnBkZikgYXMgYW4gY2xhc3MgYmFsYW5jaW5nIHRlY2huaXF1ZSBmb3IgYSBmZXcgbW9kZWxzLiAKClR1bmluZyBwYXJhbWV0ZXIgc2VhcmNoZXMgKGFrYSBoeXBlcnR1bmluZykgaXMgcGVyZm9ybWVkIHVzaW5nIHRoZSBgdHVuZUdyaWRgIHBhcmFtZXRlciBpbiB0aGUgYHRyYWluKClgIGNhbGwuIFRoZSBiZXN0IG1vZGVsIGlzIHNlbGVjdGVkIHVzaW5nIHRoZSBBVUMgb2YgdGhlIFJPQy4gSGVyZSBhcmUgdGhlIG1vZGVscyBhbmQgYSBmZXcgaW50ZXJtZWRpYXRlIHJlc3VsdHMgZm9yIGVhY2ggbW9kZWwuIEF0IHRoZSBlbmQsIEkndmUgY29tcGFyZWQgdGhlIHBlcmZvcm1hbmNlIG9mIGFsbCB0aGUgbW9kZWxzIHRvZ2V0aGVyLgoKIyMgRXh0cmVtZSBHcmFkaWVudCBCb29zdGluZyAoeGdib29zdCkKCgpgYGB7ciwgZWNobz1UUlVFfQpjdHJsIDwtIHRyYWluQ29udHJvbChtZXRob2QgPSAicmVwZWF0ZWRjdiIsCiAgICAgICAgICAgICAgICAgICAgIHJlcGVhdHMgPSA1LAogICAgICAgICAgICAgICAgICAgICB2ZXJib3NlSXRlciA9IFQsCiAgICAgICAgICAgICAgICAgICAgIGNsYXNzUHJvYnMgPSBUUlVFLAogICAgICAgICAgICAgICAgICAgICBzdW1tYXJ5RnVuY3Rpb24gPSB0d29DbGFzc1N1bW1hcnkKICAgICAgICAgICAgICAgICAgICAgIyBzYW1wbGluZyA9ICdzbW90ZScKICAgICAgICAgICAgICAgICAgICAgKQpzZXQuc2VlZCgxKQp4Z2JHcmlkIDwtIGV4cGFuZC5ncmlkKAogICAgbnJvdW5kcz1jKDIsMyw0LDUsNiw3KSwKICAgIG1heF9kZXB0aD1jKDIsMyw0LDUsNiw3KSwKICAgIGV0YT1jKDAuMywwLjUpLAogICAgZ2FtbWE9MSwKICAgIGNvbHNhbXBsZV9ieXRyZWU9MSwKICAgIG1pbl9jaGlsZF93ZWlnaHQ9MSwKICAgIHN1YnNhbXBsZT0xCikKZHVtViA8LSBkdW1teVZhcnMoZm9ybXVsYSA9IFN1cnZpdmVkfi4sZGF0YSA9IHRyYWluLmltcCkKRHRyYWluIDwtIHByZWRpY3QoZHVtVix0cmFpbi5pbXApCnhnYkZpdCA8LSB0cmFpbigKICAgIHg9RHRyYWluLAogICAgeT10cmFpbi5pbXAkU3Vydml2ZWQsCiAgICBtZXRob2QgPSAneGdiVHJlZScsCiAgICB0ckNvbnRyb2wgPSBjdHJsLAogICAgIyBtZXRyaWMgPSAiS2FwcGEiLAogICAgdHVuZUdyaWQgPSB4Z2JHcmlkLAogICAgdmVyYm9zZSA9IFRSVUUKKQp4Z2JGaXQKcGxvdCh4Z2JGaXQpCnhnYi5pbXBvcnRhbmNlKGZlYXR1cmVfbmFtZXMgPSBjb2xuYW1lcyhEdHJhaW4pLG1vZGVsID0geGdiRml0JGZpbmFsTW9kZWwpICU+JQogICAgeGdiLmdncGxvdC5pbXBvcnRhbmNlKCkKCmRlbnNpdHlwbG90KHhnYkZpdCxwY2g9J3wnKQpwcmVkaWN0KHhnYkZpdCx0eXBlID0gJ3Byb2InKSAtPiB0cmFpbi5Qcm9icwpoaXN0b2dyYW0oflN1cnZpdmVkK0RlYWQsdHJhaW4uUHJvYnMpCmBgYAoKIyMgRXh0cmVtZSBHcmFkaWVudCBCb29zdGluZyAoeGdib29zdCkgLSBTTU9URSBTYW1wbGluZwpgYGB7cn0KY3RybCA8LSB0cmFpbkNvbnRyb2wobWV0aG9kID0gInJlcGVhdGVkY3YiLAogICAgICAgICAgICAgICAgICAgICByZXBlYXRzID0gNSwKICAgICAgICAgICAgICAgICAgICAgdmVyYm9zZUl0ZXIgPSBULAogICAgICAgICAgICAgICAgICAgICBjbGFzc1Byb2JzID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgc3VtbWFyeUZ1bmN0aW9uID0gdHdvQ2xhc3NTdW1tYXJ5LAogICAgICAgICAgICAgICAgICAgICBzYW1wbGluZyA9ICdzbW90ZScKICAgICAgICAgICAgICAgICAgICAgKQp4Z2JHcmlkIDwtIGV4cGFuZC5ncmlkKAogICAgbnJvdW5kcz1jKDIsMyw0LDUsNiw3KSwKICAgIG1heF9kZXB0aD1jKDIsMyw0LDUsNiw3KSwKICAgIGV0YT1jKDAuMywwLjUpLAogICAgZ2FtbWE9MSwKICAgIGNvbHNhbXBsZV9ieXRyZWU9MSwKICAgIG1pbl9jaGlsZF93ZWlnaHQ9MSwKICAgIHN1YnNhbXBsZT0xCikKZHVtViA8LSBkdW1teVZhcnMoZm9ybXVsYSA9IFN1cnZpdmVkfi4sZGF0YSA9IHRyYWluLmltcCkKRHRyYWluIDwtIHByZWRpY3QoZHVtVix0cmFpbi5pbXApCnNldC5zZWVkKDEpCnhnYnNtb3RlRml0IDwtIHRyYWluKAogICAgeD1EdHJhaW4sCiAgICB5PXRyYWluLmltcCRTdXJ2aXZlZCwKICAgIG1ldGhvZCA9ICd4Z2JUcmVlJywKICAgIHRyQ29udHJvbCA9IGN0cmwsCiAgICAjIG1ldHJpYyA9ICJLYXBwYSIsCiAgICB0dW5lR3JpZCA9IHhnYkdyaWQsCiAgICB2ZXJib3NlID0gVFJVRQopCnhnYnNtb3RlRml0CnBsb3QoeGdic21vdGVGaXQpCnhnYi5pbXBvcnRhbmNlKGZlYXR1cmVfbmFtZXMgPSBjb2xuYW1lcyhEdHJhaW4pLG1vZGVsID0geGdic21vdGVGaXQkZmluYWxNb2RlbCkKeGdiLmltcG9ydGFuY2UoZmVhdHVyZV9uYW1lcyA9IGNvbG5hbWVzKER0cmFpbiksbW9kZWwgPSB4Z2JzbW90ZUZpdCRmaW5hbE1vZGVsKSAlPiUKeGdiLmdncGxvdC5pbXBvcnRhbmNlKCkKCmRlbnNpdHlwbG90KHhnYnNtb3RlRml0LHBjaD0nfCcpCnByZWRpY3QoeGdic21vdGVGaXQsdHlwZSA9ICdyYXcnKSAtPiB0cmFpbi5DbGFzcwpwcmVkaWN0KHhnYnNtb3RlRml0LHR5cGUgPSAncHJvYicpIC0+IHRyYWluLlByb2JzCmhpc3RvZ3JhbSh+U3Vydml2ZWQrRGVhZCx0cmFpbi5Qcm9icykKYGBgCgojIyBHcmFkaWVudCBCb29zdGluZyAoZ2JtKQoKYGBge3J9CmN0cmwgPC0gdHJhaW5Db250cm9sKG1ldGhvZCA9ICJyZXBlYXRlZGN2IiwKICAgICAgICAgICAgICAgICAgICAgcmVwZWF0cyA9IDUsCiAgICAgICAgICAgICAgICAgICAgIHZlcmJvc2VJdGVyID0gVCwKICAgICAgICAgICAgICAgICAgICAgY2xhc3NQcm9icyA9IFRSVUUsCiAgICAgICAgICAgICAgICAgICAgIHN1bW1hcnlGdW5jdGlvbiA9IHR3b0NsYXNzU3VtbWFyeSwKICAgICAgICAgICAgICAgICAgICAgYWRhcHRpdmUgPSBsaXN0KG1pbiA9IDUsIGFscGhhID0gMC4wNSwgCiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtZXRob2QgPSAiZ2xzIiwgY29tcGxldGUgPSBUUlVFKSwKICAgICAgICAgICAgICAgICAgICAgc2VhcmNoID0gJ3JhbmRvbScKICAgICAgICAgICAgICAgICAgICAgKQpnYm1HcmlkIDwtIGV4cGFuZC5ncmlkKAogICBuLnRyZWVzPWMoNTAwLDcwMCw5MDAsMTEwMCksCiAgIGludGVyYWN0aW9uLmRlcHRoPWMoMSwyLDMpLAogICBzaHJpbmthZ2U9YygwLjEsMC4wMSksCiAgIG4ubWlub2JzaW5ub2RlPTEwCikKZHVtViA8LSBkdW1teVZhcnMoZm9ybXVsYSA9IFN1cnZpdmVkfi4sZGF0YSA9IHRyYWluLmltcCkKRHRyYWluIDwtIHByZWRpY3QoZHVtVix0cmFpbi5pbXApCnNldC5zZWVkKDEpCmJvb3N0Rml0IDwtIHRyYWluKAogICAgeCA9IER0cmFpbiwKICAgIHkgPSBib29zdC50cmFpbiRTdXJ2aXZlZCwKICAgIHRyQ29udHJvbD1jdHJsLAogICAgbWV0aG9kPSdnYm0nLAogICAgdHVuZUdyaWQ9Z2JtR3JpZAopCmJvb3N0Rml0CnBsb3QoYm9vc3RGaXQpCnh5cGxvdChvb2JhZy5pbXByb3ZlfjE6MTEwMCxkYXRhPWJvb3N0Rml0JGZpbmFsTW9kZWwsYWxwaGE9LjUseGxhYiA9ICduLnRyZWVzJykKcGxvdCh2YXJJbXAoYm9vc3RGaXQpKQpkZW5zaXR5cGxvdChib29zdEZpdCxwY2g9J3wnKQpwcmVkaWN0KGJvb3N0Rml0LHR5cGUgPSAncHJvYicpIC0+IHRyYWluLlByb2JzCmhpc3RvZ3JhbSh+U3Vydml2ZWQrRGVhZCx0cmFpbi5Qcm9icykKYGBgCgojIyBSYW5kb20gRm9yZXN0IChyZikKCmBgYHtyfQphZGFwdF9jdHJsIDwtIHRyYWluQ29udHJvbChtZXRob2QgPSAicmVwZWF0ZWRjdiIsCiAgICAgICAgICAgICAgICAgICAgIHJlcGVhdHMgPSA1LAogICAgICAgICAgICAgICAgICAgICB2ZXJib3NlSXRlciA9IFQsCiAgICAgICAgICAgICAgICAgICAgIGNsYXNzUHJvYnMgPSBUUlVFLAogICAgICAgICAgICAgICAgICAgICBzdW1tYXJ5RnVuY3Rpb24gPSB0d29DbGFzc1N1bW1hcnksCiAgICAgICAgICAgICAgICAgICAgIGFkYXB0aXZlID0gbGlzdChtaW4gPSA1LCBhbHBoYSA9IDAuMDUsIAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICBtZXRob2QgPSAiZ2xzIiwgY29tcGxldGUgPSBUUlVFKSwKICAgICAgICAgICAgICAgICAgICAgc2VhcmNoID0gJ3JhbmRvbScKICAgICAgICAgICAgICAgICAgICAgKQoKcmZHcmlkIDwtIGV4cGFuZC5ncmlkKG10cnk9Yyg1LDEwLDE1LDIwLDI1KSkKc2V0LnNlZWQoMSkKcmZGaXQueSA8LSB0cmFpbigKICAgIGZvcm0gPSBTdXJ2aXZlZH4uLAogICAgZGF0YSA9IHRyYWluLmltcCwKICAgIG1ldGhvZCA9ICdyZicsCiAgICB0ckNvbnRyb2wgPSBhZGFwdF9jdHJsLAogICAgdHVuZUdyaWQgPSByZkdyaWQsCiAgICB2ZXJib3NlID0gVFJVRSwKICAgIG50cmVlID0gNDAwCikKcmZGaXQueQpwbG90KHJmRml0LnkpCnBsb3QocmZGaXQueSRmaW5hbE1vZGVsKQpkZW5zaXR5cGxvdChyZkZpdC55LHBjaD0nfCcpCnByZWRpY3QocmZGaXQueSx0eXBlID0gJ3Byb2InKSAtPiB0cmFpbi5yZi5Qcm9icwpoaXN0b2dyYW0oflN1cnZpdmVkK0RlYWQsdHJhaW4ucmYuUHJvYnMpCmBgYAoKIyMgUmFuZG9tIEZvcmVzdCAocmYpIC0gU01PVEUKCmBgYHtyfQpjdHJsIDwtIHRyYWluQ29udHJvbChtZXRob2QgPSAicmVwZWF0ZWRjdiIsCiAgICAgICAgICAgICAgICAgICAgIHJlcGVhdHMgPSA1LAogICAgICAgICAgICAgICAgICAgICB2ZXJib3NlSXRlciA9IFQsCiAgICAgICAgICAgICAgICAgICAgIGNsYXNzUHJvYnMgPSBUUlVFLAogICAgICAgICAgICAgICAgICAgICBzdW1tYXJ5RnVuY3Rpb24gPSB0d29DbGFzc1N1bW1hcnksCiAgICAgICAgICAgICAgICAgICAgIHNhbXBsaW5nID0gJ3Ntb3RlJwogICAgICAgICAgICAgICAgICAgICApCgpyZkdyaWQgPC0gZXhwYW5kLmdyaWQobXRyeT1jKDUsMTAsMTUsMjAsMjUpKQpzZXQuc2VlZCgxKQpyZnNtb3RlRml0LnkgPC0gdHJhaW4oCiAgICBmb3JtID0gU3Vydml2ZWR+LiwKICAgIGRhdGEgPSB0cmFpbi5pbXAsCiAgICBtZXRob2QgPSAncmYnLAogICAgdHJDb250cm9sID0gY3RybCwKICAgICMgbWV0cmljID0gIkthcHBhIiwKICAgIHR1bmVHcmlkID0gcmZHcmlkLAogICAgdmVyYm9zZSA9IFRSVUUsCiAgICBudHJlZSA9IDQwMAopCnJmc21vdGVGaXQueQpwbG90KHJmc21vdGVGaXQueSkKcGxvdChyZnNtb3RlRml0LnkkZmluYWxNb2RlbCkKZGVuc2l0eXBsb3QocmZzbW90ZUZpdC55LHBjaD0nfCcpCnByZWRpY3QocmZzbW90ZUZpdC55LHR5cGUgPSAncHJvYicpIC0+IHRyYWluLnJmc21vdGVGaXQuUHJvYnMKaGlzdG9ncmFtKH5TdXJ2aXZlZCtEZWFkLHRyYWluLnJmc21vdGVGaXQuUHJvYnMpCmBgYAoKIyMgRWxhc3RpbmV0CgpgYGB7cn0KY3RybCA8LSB0cmFpbkNvbnRyb2wobWV0aG9kID0gInJlcGVhdGVkY3YiLAogICAgICAgICAgICAgICAgICAgICByZXBlYXRzID0gNSwKICAgICAgICAgICAgICAgICAgICAgdmVyYm9zZUl0ZXIgPSBULAogICAgICAgICAgICAgICAgICAgICBjbGFzc1Byb2JzID0gVFJVRSwKICAgICAgICAgICAgICAgICAgICAgc3VtbWFyeUZ1bmN0aW9uID0gdHdvQ2xhc3NTdW1tYXJ5LAogICAgICAgICAgICAgICAgICAgICBzYW1wbGluZyA9ICdzbW90ZScKICAgICAgICAgICAgICAgICAgICAgKQpnbG1uZXRHcmlkIDwtIGV4cGFuZC5ncmlkKC5hbHBoYSA9IGMoMCwuMSwuMiwuNCwuNiwuOCwxKSwKICAgICAgICAgICAgICAgICAgICAgICAgICAubGFtYmRhID0gc2VxKDAuMDEsMC4yLGxlbmd0aC5vdXQgPSA0MCkpCnNldC5zZWVkKDEpCmR1bVYgPC0gZHVtbXlWYXJzKGZvcm11bGEgPSBTdXJ2aXZlZH4uLGRhdGEgPSB0cmFpbi5pbXApCkR0cmFpbiA8LSBwcmVkaWN0KGR1bVYsdHJhaW4uaW1wKQpnbG1uZXRGaXQgPC0gdHJhaW4oCiAgICB4ID0gRHRyYWluLAogICAgeSA9IHRyYWluLmltcCRTdXJ2aXZlZCwKICAgIHRyQ29udHJvbD1jdHJsLAogICAgbWV0aG9kPSdnbG1uZXQnLAogICAgdHVuZUdyaWQ9Z2xtbmV0R3JpZAopCmdsbW5ldEZpdApwbG90KGdsbW5ldEZpdCxwbG90VHlwZT0nbGV2ZWwnKQpwbG90KHZhckltcChnbG1uZXRGaXQpKQpkZW5zaXR5cGxvdChnbG1uZXRGaXQscGNoPSd8JykKcHJlZGljdChnbG1uZXRGaXQsdHlwZSA9ICdwcm9iJykgLT4gdHJhaW4uZ2xtbmV0LlByb2JzCmhpc3RvZ3JhbSh+U3Vydml2ZWQrRGVhZCx0cmFpbi5nbG1uZXQuUHJvYnMpCgpgYGAKCiMgQ29tcGFyZSBtb2RlbHMKCmBgYHtyfQpyZSA8LQogICAgcmVzYW1wbGVzKAogICAgeCA9IGxpc3QoCiAgICB4Z2IgPSB4Z2JGaXQsCiAgICB4Z2JzbW90ZSA9IHhnYnNtb3RlRml0LAogICAgcmYgPSByZkZpdC55LAogICAgcmZzbW90ZSA9IHJmc21vdGVGaXQueSwKICAgIGdibSA9IGJvb3N0Rml0LAogICAgZWxhc3RpbmV0PWdsbW5ldEZpdAogICAgKQogICAgKQpzdW1tYXJ5KHJlKQpid3Bsb3QocmUpCnN1bW1hcnkoZGlmZihyZSkpCmBgYAoKCmBgYHtyfQpzaW11bGF0ZWRUcmFpbiA8LSBkYXRhLmZyYW1lKENsYXNzID0gdHJhaW4uaW1wJFN1cnZpdmVkKQpzaW11bGF0ZWRUcmFpbiRyZiA9IHByZWRpY3QocmZGaXQueSx0eXBlID0gJ3Byb2InKVtbMV1dCnNpbXVsYXRlZFRyYWluJHJmc21vdGUgPSBwcmVkaWN0KHJmc21vdGVGaXQueSx0eXBlID0gJ3Byb2InKVtbMV1dCnNpbXVsYXRlZFRyYWluJHhnYiA9IHByZWRpY3QoeGdiRml0LHR5cGUgPSAncHJvYicpW1sxXV0Kc2ltdWxhdGVkVHJhaW4keGdic21vdGUgPSBwcmVkaWN0KHhnYnNtb3RlRml0LHR5cGUgPSAncHJvYicpW1sxXV0Kc2ltdWxhdGVkVHJhaW4kYm9vc3QgPSBwcmVkaWN0KGJvb3N0Rml0LHR5cGUgPSAncHJvYicpW1sxXV0KYGBgCgpgYGB7cn0KY2FsQ3VydmUgPC0gY2FsaWJyYXRpb24oeCA9IENsYXNzfnJmK3Jmc21vdGUreGdiK3hnYnNtb3RlK2Jvb3N0LGRhdGEgPSBzaW11bGF0ZWRUcmFpbikKeHlwbG90KGNhbEN1cnZlLGF1dG8ua2V5PWxpc3QoY29sdW1ucz0zKSkKYGBgCgojIENhbGlicmF0aW5nIHByb2JhYmlsaXRpZXMKCmBgYHtyfQpib29zdHNpZ21vaWRDYWwgPC0gZ2xtKHJlbGV2ZWwoQ2xhc3MscmVmPSdEZWFkJyl+Ym9vc3Qsc2ltdWxhdGVkVHJhaW4sZmFtaWx5ID0gJ2Jpbm9taWFsJykKY29lZihzdW1tYXJ5KGJvb3N0c2lnbW9pZENhbCkpCnNpbXVsYXRlZFRyYWluJGJvb3N0U2lnID0gcHJlZGljdChib29zdHNpZ21vaWRDYWwsdHlwZSA9ICdyZXNwb25zZScpCgp4Z2JzbW90ZXNpZ21vaWRDYWwgPC0gZ2xtKHJlbGV2ZWwoQ2xhc3MscmVmPSdEZWFkJyl+eGdic21vdGUsc2ltdWxhdGVkVHJhaW4sZmFtaWx5ID0gJ2Jpbm9taWFsJykKY29lZihzdW1tYXJ5KHhnYnNtb3Rlc2lnbW9pZENhbCkpCnNpbXVsYXRlZFRyYWluJHhnYnNtb3RlU2lnID0gcHJlZGljdCh4Z2JzbW90ZXNpZ21vaWRDYWwsdHlwZSA9ICdyZXNwb25zZScpCgpjYWxpYnJhdGlvbih4ID0gQ2xhc3N+Ym9vc3QrYm9vc3RTaWcsZGF0YSA9IHNpbXVsYXRlZFRyYWluKSAlPiUKICAgIHh5cGxvdChhdXRvLmtleT1saXN0KGNvbHVtbnM9MikpCmNhbGlicmF0aW9uKHggPSBDbGFzc354Z2JzbW90ZSt4Z2JzbW90ZVNpZyxkYXRhID0gc2ltdWxhdGVkVHJhaW4pICU+JQogICAgeHlwbG90KGF1dG8ua2V5PWxpc3QoY29sdW1ucz0yKSkKYGBgCgoKIyBFdmFsdWF0aW9uCgpgYGB7cn0KdGVzdC5pbXAgPC0gdGVzdC5yYXcKCiNFbWJhcmtlZAp0ZXN0LmltcCRFbWJhcmtlZFtpcy5uYSh0ZXN0LmltcCRFbWJhcmtlZCldPSdTJwoKI1RpdGxlCnRlc3QucmF3JHRpdGxlIDwtIHN0cl9leHRyYWN0KHBhdHRlcm4gPSAnW2EtekEtWl0rKD89XFwuKScsc3RyaW5nID0gdGVzdC5yYXckTmFtZSkKI3Rlc3QucmF3JHRpdGxlIDwtIGFzLmZhY3Rvcih0ZXN0LnJhdyR0aXRsZSkKdGVzdC5pbXAkdGl0bGUgPC0gYXMuY2hhcmFjdGVyKHRlc3QucmF3JHRpdGxlKQp0ZXN0LmltcCR0aXRsZVt0ZXN0LmltcCR0aXRsZSAlaW4lIGMoJ0NhcHQnLCdDb2wnLCdNYWpvcicpXSA8LSAnT2ZmaWNlcicKdGVzdC5pbXAkdGl0bGVbdGVzdC5pbXAkdGl0bGUgJWluJSBjKCdEb24nLCdEcicsJ1JldicsJ1NpcicsJ0pvbmtoZWVyJywnQ291bnRlc3MnLCdMYWR5JywnRG9uYScpXSA8LSAnUm95YWx0eScKdGVzdC5pbXAkdGl0bGVbdGVzdC5pbXAkdGl0bGUgJWluJSBjKCdNcnMnLCdNbWUnKV0gPC0gJ01ycycKdGVzdC5pbXAkdGl0bGVbdGVzdC5pbXAkdGl0bGUgJWluJSBjKCdNcycsJ01sbGUnKV0gPC0gJ01pc3MnCnRlc3QuaW1wJHRpdGxlIDwtIGFzLmZhY3Rvcih0ZXN0LmltcCR0aXRsZSkKCiNNaXNzaW5nIGFnZQptaXNzaW5nLmFnZSA8LSB0ZXN0LmltcCAlPiUgZmlsdGVyKGlzLm5hKEFnZSkpCmFnZS5wcmVkaWN0ZWQgPC0gcHJlZGljdChyZkZpdCwgbmV3ZGF0YSA9IG1pc3NpbmcuYWdlKQp0ZXN0LmltcCRBZ2VbaXMubmEodGVzdC5pbXAkQWdlKV0gPC0gYWdlLnByZWRpY3RlZAp0ZXN0LmltcCRBZ2VbdGVzdC5pbXAkdGl0bGU9PSdNYXN0ZXInICYgdGVzdC5pbXAkQWdlID4gMjBdIDwtIDQKCiNDaGlsZAp0ZXN0LmltcCRjaGlsZCA8LSAwCnRlc3QuaW1wJGNoaWxkW3Rlc3QuaW1wJEFnZTwxOF0gPC0gMQp0ZXN0LmltcCRhbG1vc3RhZHVsdCA8LSBhcy5udW1lcmljKGJldHdlZW4odGVzdC5pbXAkQWdlLDE2LDE4KSkKCiNZb3VuZy9vbGQKdGVzdC5pbXAkWW91bmcgPC0gaWZlbHNlKHRlc3QuaW1wJEFnZTwxMCwxLDApCnRlc3QuaW1wJFNlbmlvcnMgPC0gaWZlbHNlKHRlc3QuaW1wJEFnZT42MCwxLDApCgojRmFtaWx5IFJlbGF0ZWQKdGVzdC5pbXAkVG90YWxGYW0gPC0gdGVzdC5pbXAkU2liU3AgKyB0ZXN0LmltcCRQYXJjaCArIDEKdGVzdC5pbXAkTmFtZSA8LSBOVUxMCnRlc3QuaW1wJExhcmdlUGFyQ2ggPC0gYXMubnVtZXJpYyh0ZXN0LmltcCRQYXJjaD49MykKdGVzdC5pbXAkTGFyZ2VTaWJTcCA8LSBhcy5udW1lcmljKHRlc3QuaW1wJFNpYlNwPj0zKQp0ZXN0LmltcCRTaW5nbGUgPC0gaWZlbHNlKHRlc3QuaW1wJFRvdGFsRmFtPT0xLDEsMCkKdGVzdC5pbXAkQ291cGxlIDwtIGlmZWxzZSh0ZXN0LmltcCRUb3RhbEZhbT09MiwxLDApCnRlc3QuaW1wJEZhbWlseSA8LSBpZmVsc2UodGVzdC5pbXAkVG90YWxGYW0+MiwxLDApCgojQ2FiaW4gJiBEZWNrCnRlc3QuaW1wJENhYmluTWlzc2luZyA8LSBhcy5udW1lcmljKGlzLm5hKHRlc3QucmF3JENhYmluKSkKdGVzdC5pbXAkQ2FiaW5Db2RlIDwtIG1hcF9jaHIodGVzdC5yYXckQ2FiaW4sfnN0cl9zcGxpdChzdHJpbmcgPSAueCxwYXR0ZXJuID0gJycpW1sxXV1bMV0pCnRlc3QuaW1wJENhYmluQ29kZVtpcy5uYSh0ZXN0LmltcCRDYWJpbkNvZGUpXSA8LSAnVScKdGVzdC5pbXAkQ2FiaW5OdW0gPC0gYXMubnVtZXJpYyhtYXBfY2hyKHRlc3QucmF3JENhYmluLH5zdHJfc3BsaXQoc3RyaW5nID0gLngscGF0dGVybiA9ICdbYS16QS1aXScpW1sxXV1bMl0pKQp0ZXN0LmltcCRDYWJpbk51bSA8LSBtYXBfaW50KHRlc3QuaW1wJENhYmluTnVtLCB+YXMuaW50ZWdlcihzdHJfc3BsaXQoLngscGF0dGVybiA9ICcnLHNpbXBsaWZ5ID0gVClbMV1bMV0pKQp0ZXN0LmltcCRDYWJpbk51bVtpcy5uYSh0ZXN0LmltcCRDYWJpbk51bSldIDwtIDAKCnRlc3QuaW1wJENhYmluQ29kZSA8LSBmYWN0b3IoCiAgICB4ID0gdGVzdC5pbXAkQ2FiaW5Db2RlLAogICAgbGV2ZWxzID0gdW5pcXVlKHRyYWluLmltcCRDYWJpbkNvZGUpCikKCnRlc3QuaW1wJFRvcERlY2sgPC0gaWZlbHNlKHRlc3QuaW1wJENhYmluQ29kZSAlaW4lIGMoJ0EnLCdCJyksMSwwKQp0ZXN0LmltcCRNaWREZWNrIDwtIGlmZWxzZSh0ZXN0LmltcCRDYWJpbkNvZGUgJWluJSBjKCdDJywnRCcpLDEsMCkKdGVzdC5pbXAkTG93ZXJEZWNrIDwtIGlmZWxzZSh0ZXN0LmltcCRUb3BEZWNrPT0wICYgdGVzdC5pbXAkTWlkRGVjayA9PTAgLDEsMCkKCnRlc3QuaW1wJE51bWJlcm9mQ2FiaW5zIDwtIG1hcF9pbnQodGVzdC5yYXckQ2FiaW4sfnN0cl9zcGxpdChzdHJpbmcgPSAueCxwYXR0ZXJuID0gJyAnKVtbMV1dICU+JSBsZW5ndGgpCnRlc3QuaW1wJENhYmluIDwtIE5VTEwKCgojIFRpY2tldAp0ZXN0LmltcCAlPD4lCiAgICBtdXRhdGUoCiAgICAgIFRpY2tldCA9IHN0cl90b191cHBlcihUaWNrZXQpICU+JQogICAgICAgICAgc3RyX3JlcGxhY2VfYWxsKHBhdHRlcm4gPSByZWdleChwYXR0ZXJuID0gJ1suXFwvXScpLHJlcGxhY2VtZW50ID0gJycpLAogICAgICBUaWNrZXROdW0gPSBzdHJfZXh0cmFjdChUaWNrZXQscGF0dGVybiA9IHJlZ2V4KCcoWzAtOV0pezMsfScpKSwKICAgICAgVGlja2V0TnVtU3RhcnQgPSBtYXBfaW50KFRpY2tldE51bSx+YXMuaW50ZWdlcihzdHJfc3BsaXQoLngscGF0dGVybiA9ICcnLHNpbXBsaWZ5ID0gVClbMV0pKSwKICAgICAgVGlja2V0TnVtTGVuID0gbWFwX2ludChUaWNrZXROdW0sfmRpbShzdHJfc3BsaXQoLngscGF0dGVybiA9ICcnLHNpbXBsaWZ5ID0gVCkpWzJdKSwKICAgICAgVGlja2V0Q2hhciA9IHN0cl9leHRyYWN0KFRpY2tldCxwYXR0ZXJuID0gcmVnZXgoJ15bYS16QS1aL1xcLl0rJykpCiAgICAgICkgJT4lCiAgICBtdXRhdGUoCiAgICAgICAgVGlja2V0Q2hhciA9IG1hcF9jaHIoLng9VGlja2V0Q2hhciwKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAuZj1+c3RyX3NwbGl0KHN0cmluZz0ueCwgcGF0dGVybiA9ICcnLHNpbXBsaWZ5ID0gVClbMV0pCiAgICAgICAgKSAlPiUgIAogICAgbXV0YXRlKAogICAgICBUaWNrZXRDaGFyID0gaWZlbHNlKGlzLm5hKFRpY2tldENoYXIpLCdVJyxUaWNrZXRDaGFyKSwKICAgICAgVGlja2V0TnVtU3RhcnQgPSBpZmVsc2UoaXMubmEoVGlja2V0TnVtU3RhcnQpLDAsVGlja2V0TnVtU3RhcnQpLAogICAgICBUaWNrZXROdW1MZW4gPSBpZmVsc2UoaXMubmEoVGlja2V0TnVtTGVuKSwwLFRpY2tldE51bUxlbiksCiAgICApCnRlc3QuaW1wJFRpY2tldCA8LSBOVUxMCnRlc3QuaW1wJFRpY2tldE51bSA8LSBOVUxMCgojRmFyZQp0ZXN0LmltcCRGYXJlW2lzLm5hKHRlc3QuaW1wJEZhcmUpXSA8LSAxNC40NTQyCnRlc3QuaW1wJEZhcmVbdGVzdC5pbXAkRmFyZT4yMzJdIDwtIDIzMgoKYGBgCgoKCmBgYHtyfQpkdW1WIDwtIGR1bW15VmFycyhmb3JtdWxhID0gfi4sZGF0YSA9IHRlc3QuaW1wKQpEdGVzdCA8LSBwcmVkaWN0KGR1bVYsdGVzdC5pbXApCgpib29zdFByZWQgPC0gaWZlbHNlKHByZWRpY3QoYm9vc3RGaXQkZmluYWxNb2RlbCxuZXdkYXRhID0gRHRlc3Qsbi50cmVlcyA9IGJvb3N0Rml0JGJlc3RUdW5lJG4udHJlZXMsdHlwZSA9ICdyZXNwb25zZScpPjAuNSwxLDIpCnhnYlByZWQgPC0gcHJlZGljdChvYmplY3QgPSB4Z2JGaXQsIG5ld2RhdGEgPSBEdGVzdCkKeGdic21vdGVQcmVkIDwtIHByZWRpY3Qob2JqZWN0ID0geGdic21vdGVGaXQsIG5ld2RhdGEgPSBEdGVzdCkKcmZQcmVkIDwtIHByZWRpY3Qob2JqZWN0ID0gcmZGaXQueSwgbmV3ZGF0YSA9IHRlc3QuaW1wKQpyZnNtb3RlUHJlZCA8LSBwcmVkaWN0KG9iamVjdCA9IHJmc21vdGVGaXQueSwgbmV3ZGF0YSA9IHRlc3QuaW1wKQoKYm9vc3RQcm9iIDwtIGRhdGEuZnJhbWUoYm9vc3Q9cHJlZGljdChib29zdEZpdCRmaW5hbE1vZGVsLG5ld2RhdGEgPSBEdGVzdCxuLnRyZWVzID0gYm9vc3RGaXQkYmVzdFR1bmUkbi50cmVlcyx0eXBlID0gJ3Jlc3BvbnNlJykpCmJvb3N0U2lnUHJlZCA8LSBwcmVkaWN0KG9iamVjdCA9IGJvb3N0c2lnbW9pZENhbCxuZXdkYXRhID0gYm9vc3RQcm9iLHR5cGUgPSAncmVzcG9uc2UnKQpib29zdFNpZ1ByZWQgPC0gaWZlbHNlKGJvb3N0U2lnUHJlZD49MC41LDEsMikKCnhnYnNtb3Rlc2lnbW9pZENhbAp4Z2JQcm9iIDwtIGRhdGEuZnJhbWUoeGdic21vdGU9cHJlZGljdChvYmplY3QgPSB4Z2JzbW90ZUZpdCwgbmV3ZGF0YSA9IER0ZXN0LHR5cGUgPSAncHJvYicpW1sxXV0pCnhnYnNtb3RlU2lnUHJlZCA8LSBwcmVkaWN0KG9iamVjdCA9IHhnYnNtb3Rlc2lnbW9pZENhbCxuZXdkYXRhID0geGdiUHJvYix0eXBlID0gJ3Jlc3BvbnNlJykKeGdic21vdGVTaWdQcmVkIDwtIGlmZWxzZSh4Z2JzbW90ZVNpZ1ByZWQ+MC41LDEsMikKYGBgCgoKCmBgYHtyfQpQSUQgPC0KICAgIHJlYWREYXRhKFRpdGFuaWMucGF0aCwKICAgIHRlc3QuZGF0YS5maWxlLAogICAgdGVzdC5jb2x1bW4udHlwZXMsCiAgICBtaXNzaW5nLnR5cGVzKQpQSUQgPC0gUElEJFBhc3NlbmdlcklkCmBgYAoKYGBge3J9CmZvcihtIGluIGxzKHBhdHRlcm4gPSAnUHJlZCcpKSB7CiAgICB3cml0ZS5jc3YoCiAgICB4ID0gZGF0YS5mcmFtZSgKICAgIFBhc3NlbmdlcklkID0gUElELAogICAgU3Vydml2ZWQgPSBhcy5udW1lcmljKGV2YWwocGFyc2UodGV4dCA9IG0pKSkgKiAtMSArIDIKICAgICksCiAgICBmaWxlID0gcGFzdGUwKG0sJy5jc3YnKSwKICAgIHJvdy5uYW1lcyA9IEYKICAgICkKfQoKYGBgCgojIENvbmNsdXNpb25zCg==